-- Logs begin at Mon 2022-08-22 10:15:07 UTC, end at Mon 2022-08-22 11:24:26 UTC. -- Aug 22 10:44:41.221369 dv-cinder-853941-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 22 10:44:41.229555 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 22 10:44:41 2022] *** Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: nodename: dv-cinder-853941-1-iscsi Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: machine: x86_64 Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: clock source: unix Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: pcre jit disabled Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: detected number of CPU cores: 4 Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: current working directory: / Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: detected binary path: /usr/bin/uwsgi-core Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: your processes number limit is 47680 Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: your memory page size is 4096 bytes Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: detected max file descriptor number: 2048 Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: lock engine: pthread robust mutexes Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: thunder lock: enabled Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 22 10:44:41.234371 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 22 10:44:41.275970 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: Python main interpreter initialized at 0x56023fe8b9c0 Aug 22 10:44:41.275970 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: python threads support enabled Aug 22 10:44:41.275970 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: your server socket listen backlog is limited to 100 connections Aug 22 10:44:41.275970 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: your mercy for graceful operations on workers is 90 seconds Aug 22 10:44:41.276364 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: mapped 403077 bytes (393 KB) for 2 cores Aug 22 10:44:41.276440 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: *** Operational MODE: preforking *** Aug 22 10:44:41.276682 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: *** uWSGI is running in multiple interpreter mode *** Aug 22 10:44:41.276682 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: spawned uWSGI master process (pid: 81519) Aug 22 10:44:41.276896 dv-cinder-853941-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 22 10:44:41.278546 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: spawned uWSGI worker 1 (pid: 81529, cores: 1) Aug 22 10:44:41.278546 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: spawned uWSGI worker 2 (pid: 81530, cores: 1) Aug 22 10:44:41.282252 dv-cinder-853941-1-iscsi devstack@keystone.service[81519]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 22 10:44:45.619310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 10:44:45.619310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: from cryptography.utils import int_from_bytes Aug 22 10:44:45.620170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 10:44:45.620170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: from cryptography.utils import int_from_bytes Aug 22 10:44:45.620581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 10:44:45.620581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: from cryptography.utils import int_from_bytes Aug 22 10:44:45.620698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 22 10:44:45.620698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: from cryptography.utils import int_from_bytes Aug 22 10:44:45.670437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 10:44:45.670606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 10:44:45.670800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] command line args: [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 10:44:45.671061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 10:44:45.671230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 22 10:44:45.671314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ================================================================================ {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 10:44:45.671580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 22 10:44:45.671678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] admin_token = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.671756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] command line args: [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 22 10:44:45.671832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] config_dir = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.671913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.671990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 22 10:44:45.672094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] config_source = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.672178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ================================================================================ {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 22 10:44:45.672274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] debug = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.672363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] admin_token = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.672471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.672735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] config_dir = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.672735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.672848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.672915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] config_source = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] debug = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] list_limit = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] log_config_append = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.673991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] log_dir = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] list_limit = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] log_file = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] log_config_append = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.674929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] log_dir = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] log_file = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.675913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.676983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.677088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.677251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.677369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.677485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.677588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.677760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.677877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.678059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.678871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] publish_errors = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.679917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] publish_errors = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] standard_threads = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.680909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] standard_threads = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] use_journal = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] use_json = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] use_stderr = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.681937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.682049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] use_syslog = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.682161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.682300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.682422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] use_journal = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.682422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] use_json = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.682639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.682750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] use_stderr = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.682867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.682990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] use_syslog = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.683109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.683219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 22 10:44:45.683332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.683499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.683617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.683802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.683911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.684953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.685894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.686896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687136 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.687992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.688914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.689909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690560 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.690907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.691964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.692986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.693083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.693178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.693271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.693378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.693528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.693721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.693854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.694594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.694720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.694856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.694990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.695162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.695422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.695524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.695524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.695741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.695854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.695977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.696911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697365 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.697968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.698935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.699980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.700901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.701895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.702010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.702124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.702245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.706954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.707935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.708944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.709980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] auth.external = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.710955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] auth.password = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] auth.token = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] auth.external = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.711930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] auth.password = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] credential.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] auth.token = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.712940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] credential.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713762 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.713980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714420 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.714951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.715963 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.716988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.717991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.718944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.719894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.720866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721556 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.721948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.722029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.722187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.722290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.722502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.722734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.722883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.723958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.724089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.724218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.724347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.724475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.724609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.724750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.724880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.725929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.726058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.726187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.726355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.726489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.726618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.726752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.726898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727296 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.727946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.728964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729651 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.729989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.730086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.730191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.730338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.733968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.734994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.735985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.736974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.737990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] role.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.738887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] role.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.739955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.740035 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.740205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.740282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.740450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.740539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.740685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.741265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.741457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.741527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.741746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.741844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.741949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.742955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] role.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] role.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.743920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.744894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.745904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.746054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.746134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.746305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.746390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.746557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.746636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.746816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.747226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.747326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.747473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.747552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.747727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.747806 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.747978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.748999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.749953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.750109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.750190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.750376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.750544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.750666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.755947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.756046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.756130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.756202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.756274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.756390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.756981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.connection = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.757961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.758941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759173 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.759903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.connection = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81530) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.760559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 22 10:44:45.761418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81529) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 22 10:44:45.786765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.786998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.787224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.790786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.791053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.802087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.802180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.802276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.830448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.831234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.831234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.831395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.889089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.889972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.943663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.943979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.944403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:45.944764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.107673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.107948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.108954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.109174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.139269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.139443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.139666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.139784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.139869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.140919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.141137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.152368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.152775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.165477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.165712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.165804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.165932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166736 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81530) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.166957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.167175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81529) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 22 10:44:46.187954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.188167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 10:44:46.188489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.188581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.188690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 10:44:46.188772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 10:44:46.188967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.189048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.189228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 22 10:44:46.189309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.189485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.189565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.189761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.189990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.190071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.190554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.200777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.201058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.201330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.201617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.201700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.201975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.202266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.202556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.208827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.209577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.211823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.212114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.212402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.212562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.212688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.212856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.212977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.213145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.213246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.213439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.213520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.213731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.213812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.214001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.214161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.214302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.214592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.214702 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.214909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.215016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.215250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.215527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.238026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.238133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.239519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.239631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.239809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.239894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.245024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.245250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.245350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.245540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.245623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.245829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.252231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.252525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.252610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.252814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.257217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.257510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.257594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.257800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.257884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.258089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.258170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.258391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.258487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.258674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.258758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.258949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.259089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.259226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.259367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.259639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.276585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.276870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.277619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.277904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.280763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.281857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.284010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.284314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.285081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.285479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.288568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.289824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.291756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.293144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.295025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.295339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.295631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.295922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.296216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.296479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.296572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.296785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.297076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.297364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.297646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.297926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.312904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.313198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.327616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.327848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.328160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.328419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.328624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.338134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.338446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.338736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.339022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.339309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.339599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.339893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.340483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.340770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.341053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.341336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.341622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.341901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.342190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.358617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.360588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.361686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.364611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.365590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.366085 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.367596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.367884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.368186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.370479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.373433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.374962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.375824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.376246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.376419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.376679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.379323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.379634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.379926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.380218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.387721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.388603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.388895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.389186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.390105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.393285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.393581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.393876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.394163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.396080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81530) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.396371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.396656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.397475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.397902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.398366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.398651 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.398928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.399214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.399519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.399800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.400068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.400357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.400622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81530) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.406343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.406774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.407187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.417609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81529) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 22 10:44:46.417901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.418192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.418533 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.418955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.419362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.419841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.424831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.424831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.424831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.424831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.424831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.424831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.425204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81529) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 22 10:44:46.437511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56023fe8b9c0 pid: 81530 (default app) Aug 22 10:44:46.441427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636d18d-9686-4810-badb-655994570f2a None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:46.441725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636d18d-9686-4810-badb-655994570f2a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:46.441997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636d18d-9686-4810-badb-655994570f2a None None] PATH_INFO: `/v3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:46.446344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Mon Aug 22 10:44:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:44:46.463555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56023fe8b9c0 pid: 81529 (default app) Aug 22 10:44:53.824257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339528fe-1d18-4b1f-8f26-3729b60a3f46 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:53.824775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339528fe-1d18-4b1f-8f26-3729b60a3f46 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:53.824775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339528fe-1d18-4b1f-8f26-3729b60a3f46 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:53.826088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 2/2] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:44:53.835177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:53.835588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:53.835724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:53.906199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: INFO dbcounter [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] Registered counter for database keystone Aug 22 10:44:53.925413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG oslo_db.sqlalchemy.engines [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81529) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 10:44:53.932202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writer thread running {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 10:44:54.114525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG passlib.handlers.bcrypt [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81529) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 10:44:54.114865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG passlib.handlers.bcrypt [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81529) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 10:44:54.169584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:44:54.171225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:44:54.301647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1/3] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:44:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 22 10:44:54.714454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339528fe-1d18-4b1f-8f26-3729b60a3f46 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:44:54.782773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: INFO dbcounter [None req-339528fe-1d18-4b1f-8f26-3729b60a3f46 None None] Registered counter for database keystone Aug 22 10:44:54.805064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG oslo_db.sqlalchemy.engines [None req-339528fe-1d18-4b1f-8f26-3729b60a3f46 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81530) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 22 10:44:54.811247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writer thread running {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 22 10:44:54.931114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339528fe-1d18-4b1f-8f26-3729b60a3f46 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:44:54.935375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95663eea-ad9b-4291-bea2-b5e6367c9153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:44:54.936007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95663eea-ad9b-4291-bea2-b5e6367c9153 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:54.936282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95663eea-ad9b-4291-bea2-b5e6367c9153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:54.936547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95663eea-ad9b-4291-bea2-b5e6367c9153 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:54.945150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95663eea-ad9b-4291-bea2-b5e6367c9153 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:44:54.996607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING keystone.common.rbac_enforcer.enforcer [None req-95663eea-ad9b-4291-bea2-b5e6367c9153 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 10:44:54.996892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95663eea-ad9b-4291-bea2-b5e6367c9153 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:44:55.007283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 3/4] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 22 10:44:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 10:44:55.013336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:44:55.028517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548064b4-8960-487b-952b-fe8989e2d6e4 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:44:55.032906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b10f53-9a5d-48fd-bf38-fd75f77ce023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:44:55.034011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b10f53-9a5d-48fd-bf38-fd75f77ce023 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:55.034327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b10f53-9a5d-48fd-bf38-fd75f77ce023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:55.034600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b10f53-9a5d-48fd-bf38-fd75f77ce023 admin admin] PATH_INFO: `/v3/projects/ce1b87adbe93482a946d0fcf7c513809` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:55.036032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b10f53-9a5d-48fd-bf38-fd75f77ce023 admin admin] RBAC: Authorizing `identity:get_project(project_id=ce1b87adbe93482a946d0fcf7c513809)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:44:55.089368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d8b10f53-9a5d-48fd-bf38-fd75f77ce023 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 22 10:44:55.089704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b10f53-9a5d-48fd-bf38-fd75f77ce023 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:44:55.094190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 2/5] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Aug 22 10:44:55 2022] GET /identity/v3/projects/ce1b87adbe93482a946d0fcf7c513809 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:44:57.003412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e94fc-73b3-44b4-8154-467616bdaf78 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:57.003905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e94fc-73b3-44b4-8154-467616bdaf78 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:57.003987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e94fc-73b3-44b4-8154-467616bdaf78 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:57.008291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 4/6] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:44:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:44:57.013626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deb960-3d51-4ff4-9f8b-c1ffb304e175 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:57.014053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deb960-3d51-4ff4-9f8b-c1ffb304e175 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:57.014176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69deb960-3d51-4ff4-9f8b-c1ffb304e175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:57.034090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-69deb960-3d51-4ff4-9f8b-c1ffb304e175 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:44:57.035623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-69deb960-3d51-4ff4-9f8b-c1ffb304e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:44:57.043841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 3/7] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:44:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 22 10:44:57.309370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e94fc-73b3-44b4-8154-467616bdaf78 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:44:57.321645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e94fc-73b3-44b4-8154-467616bdaf78 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:44:57.325833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24a4f1-5cf5-4bb7-887f-952becc25a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:44:57.326455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24a4f1-5cf5-4bb7-887f-952becc25a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:57.326738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24a4f1-5cf5-4bb7-887f-952becc25a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:57.327013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24a4f1-5cf5-4bb7-887f-952becc25a2a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:57.334892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24a4f1-5cf5-4bb7-887f-952becc25a2a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:44:57.336657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24a4f1-5cf5-4bb7-887f-952becc25a2a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:44:57.345283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 5/8] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 22 10:44:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 10:44:57.350928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deb960-3d51-4ff4-9f8b-c1ffb304e175 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:44:57.359783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69deb960-3d51-4ff4-9f8b-c1ffb304e175 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:44:57.364027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479ea55-f9c0-44f4-909d-4bef9a4df7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:44:57.364644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479ea55-f9c0-44f4-909d-4bef9a4df7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:57.364922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479ea55-f9c0-44f4-909d-4bef9a4df7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:57.365735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479ea55-f9c0-44f4-909d-4bef9a4df7f1 admin admin] PATH_INFO: `/v3/users/5f27905b35c14e3c9541d9819a2b2185` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:57.367050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479ea55-f9c0-44f4-909d-4bef9a4df7f1 admin admin] RBAC: Authorizing `identity:get_user(user_id=5f27905b35c14e3c9541d9819a2b2185)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:44:57.368896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479ea55-f9c0-44f4-909d-4bef9a4df7f1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:44:57.370416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 4/9] 10.222.0.48 () {56 vars in 1198 bytes} [Mon Aug 22 10:44:57 2022] GET /identity/v3/users/5f27905b35c14e3c9541d9819a2b2185 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:44:59.319040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfcc20-75c6-410a-a035-b8cec57f922a None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:59.319580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfcc20-75c6-410a-a035-b8cec57f922a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:59.319580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfcc20-75c6-410a-a035-b8cec57f922a None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:59.320947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 6/10] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:44:59.326243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b8121-1c67-4b12-8e8d-da8d2f0a3608 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:59.326640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b8121-1c67-4b12-8e8d-da8d2f0a3608 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:59.326819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b8121-1c67-4b12-8e8d-da8d2f0a3608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:59.340495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4eae-3653-4793-9404-6403b381d838 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:59.340789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4eae-3653-4793-9404-6403b381d838 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:59.341062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4eae-3653-4793-9404-6403b381d838 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:59.342541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 7/11] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:44:59.346095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b92b8121-1c67-4b12-8e8d-da8d2f0a3608 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:44:59.347333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b92b8121-1c67-4b12-8e8d-da8d2f0a3608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:44:59.347519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:59.347804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:59.348080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:59.390252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG passlib.handlers.bcrypt [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81530) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 22 10:44:59.390770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG passlib.handlers.bcrypt [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81530) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 22 10:44:59.429474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 5/12] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:44:59.436041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:44:59.437639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:44:59.533881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 8/13] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:44:59.704280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b8121-1c67-4b12-8e8d-da8d2f0a3608 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:44:59.727925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:44:59.797068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:44:59.807262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b8121-1c67-4b12-8e8d-da8d2f0a3608 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:44:59.840394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:44:59.841033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:59.841314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:59.841587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:59.853282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:44:59.863235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:44:59.864105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:44:59.864840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 6/14] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:44:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:44:59.869450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:44:59.907721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5281c-b8b3-4e0c-b064-69de46bb980e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:44:59.910000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:44:59.946181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:44:59.947127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:44:59.947413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:44:59.947683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:44:59.950741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:44:59.969246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df672-31eb-4bf4-adb1-d987a254a409 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:00.003309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:00.003918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:00.004195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:00.004466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:00.005123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:00.005420 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:00.007172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:00.015159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:00.016669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:00.018680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 7/15] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:44:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:00.024753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:00.116009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 9/16] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 10:45:00.121277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:00.122535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be79a8-f380-4430-8c46-0033212d48be None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:00.124209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:00.156627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:00.157268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:00.157550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:00.157825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] PATH_INFO: `/v3/users/5f27905b35c14e3c9541d9819a2b2185` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:00.158983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] RBAC: Authorizing `identity:get_user(user_id=5f27905b35c14e3c9541d9819a2b2185)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:00.160434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:00.162104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 8/17] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:00 2022] GET /identity/v3/users/5f27905b35c14e3c9541d9819a2b2185 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:00.167671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:00.182052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:00.221496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131b3d7-ae01-49f8-993e-cbf3f9c98db5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:00.258352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:00.259203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:00.259490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:00.259766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:00.265809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f530e-4a54-4d11-b3d4-53f4989fa631 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:00.278586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:00.280101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:45:00.298939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:00.299834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:45:00.300614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:45:00.300614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:45:00.300614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:00.300614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 10/18] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:00.300810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:00.301008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:00.301277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:00.301549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:00.303361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:00.304957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:00.306506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 9/19] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:00.307014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:00.311899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:00.365816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:00.392102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:00.407735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c82de-6a8a-490e-91b4-1cdce82519cf None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:00.410444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae829672-65b4-42f6-8ef1-c3310fa0cafe None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:00.441659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfe6b6-4583-4fe5-903c-386290f266af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:00.442296 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfe6b6-4583-4fe5-903c-386290f266af None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:00.442582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfe6b6-4583-4fe5-903c-386290f266af None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:00.442856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfe6b6-4583-4fe5-903c-386290f266af None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:00.443669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfe6b6-4583-4fe5-903c-386290f266af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:00.443993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfe6b6-4583-4fe5-903c-386290f266af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:00.444911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909b875-08fb-4a99-b021-a2550d682e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:00.445364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfe6b6-4583-4fe5-903c-386290f266af None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:00.445516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b875-08fb-4a99-b021-a2550d682e9d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:00.445797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b875-08fb-4a99-b021-a2550d682e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:00.446072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909b875-08fb-4a99-b021-a2550d682e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:00.446821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b875-08fb-4a99-b021-a2550d682e9d None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.domain.id=default, user.id=5f27905b35c14e3c9541d9819a2b2185) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:00.447290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b875-08fb-4a99-b021-a2550d682e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:00.448726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909b875-08fb-4a99-b021-a2550d682e9d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:00.455727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 22 10:45:00 2022] GET /identity/v3/role_assignments?user.id=5f27905b35c14e3c9541d9819a2b2185&scope.domain.id=default&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:00.466752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:02.573285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88529-ed18-4dfd-891d-f3763b81c580 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:02.574251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88529-ed18-4dfd-891d-f3763b81c580 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:02.574777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b88529-ed18-4dfd-891d-f3763b81c580 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:02.576300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:02.587779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f779d2a-2579-468c-8da1-15305fa4df78 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:02.588218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f779d2a-2579-468c-8da1-15305fa4df78 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:02.588327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f779d2a-2579-468c-8da1-15305fa4df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:02.615448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8f779d2a-2579-468c-8da1-15305fa4df78 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:02.616974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8f779d2a-2579-468c-8da1-15305fa4df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:02.624958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b7515-858a-4fbf-a500-d57a44fb23ec None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:02.625514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b7515-858a-4fbf-a500-d57a44fb23ec None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:02.626007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b7515-858a-4fbf-a500-d57a44fb23ec None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:02.627882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 12/23] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:02.633716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fcdc2-0795-4df6-b5bf-60cb69487cd5 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:02.634212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fcdc2-0795-4df6-b5bf-60cb69487cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:02.634709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fcdc2-0795-4df6-b5bf-60cb69487cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:02.671217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a47fcdc2-0795-4df6-b5bf-60cb69487cd5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:02.672834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a47fcdc2-0795-4df6-b5bf-60cb69487cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:02.729885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 12/24] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:02.797678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 13/25] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:02.936421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f807-ed27-413d-bb3b-5793f6c365d8 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:02.937039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f807-ed27-413d-bb3b-5793f6c365d8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:02.937543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96f807-ed27-413d-bb3b-5793f6c365d8 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:02.939124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 13/26] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:02.950441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87abc4-d3cb-485c-aa6b-65220f22893b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:02.950980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87abc4-d3cb-485c-aa6b-65220f22893b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:02.951444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87abc4-d3cb-485c-aa6b-65220f22893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:02.976223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ad87abc4-d3cb-485c-aa6b-65220f22893b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:02.977716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ad87abc4-d3cb-485c-aa6b-65220f22893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:03.067555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 14/27] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:03.248078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f807-ed27-413d-bb3b-5793f6c365d8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:03.311429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:03.333989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87abc4-d3cb-485c-aa6b-65220f22893b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:03.337591 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:03.412670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96f807-ed27-413d-bb3b-5793f6c365d8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:03.481279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:03.484204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:03.484204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:03.484204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:03.500840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:03.502441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:45:03.518916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:03.519865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:45:03.520673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:45:03.520673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:45:03.520673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:03.520673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 14/28] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:03.526951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:03.583348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87abc4-d3cb-485c-aa6b-65220f22893b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:03.629653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189490b6-0be3-4b0b-b38b-21218d2e0b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:03.631382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189490b6-0be3-4b0b-b38b-21218d2e0b75 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:03.631806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189490b6-0be3-4b0b-b38b-21218d2e0b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:03.632090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189490b6-0be3-4b0b-b38b-21218d2e0b75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:03.632864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:03.637020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ea6b5-e88b-43f5-9dc1-a0ef433b60ba None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:03.646609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189490b6-0be3-4b0b-b38b-21218d2e0b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-189490b6-0be3-4b0b-b38b-21218d2e0b75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:03.653797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:03.654878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:03.655816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:03.657220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 15/29] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:03.662307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683a27f-9917-42ba-974b-9febbaca4508 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:03.662307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683a27f-9917-42ba-974b-9febbaca4508 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:03.662307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683a27f-9917-42ba-974b-9febbaca4508 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:03.662569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 16/30] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:03.663317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683a27f-9917-42ba-974b-9febbaca4508 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:03.681104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:03.681901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:03.682418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:03.682824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:03.683665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:03.684081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:03.685652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:03.699037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 15/31] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:03.699820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:03.743998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:03.797509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683a27f-9917-42ba-974b-9febbaca4508 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:03.813560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e795-5c56-4d76-9939-a5ffaae6bdcf None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:03.842727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:03.843359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:03.843879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:03.844170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:03.844870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:03.846983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:03.854103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b59ab-dab2-4e2d-bbda-2a220d804639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:03.855201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b59ab-dab2-4e2d-bbda-2a220d804639 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:03.855704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b59ab-dab2-4e2d-bbda-2a220d804639 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:03.856178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b59ab-dab2-4e2d-bbda-2a220d804639 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:03.857067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b59ab-dab2-4e2d-bbda-2a220d804639 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:03.857852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b59ab-dab2-4e2d-bbda-2a220d804639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:03.859890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b59ab-dab2-4e2d-bbda-2a220d804639 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:03.873586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 16/32] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:03.875295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c99ccb-4045-49ba-9ed1-aeafa6254efe None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:03.875589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c99ccb-4045-49ba-9ed1-aeafa6254efe None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:03.875860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c99ccb-4045-49ba-9ed1-aeafa6254efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:03.895832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-24c99ccb-4045-49ba-9ed1-aeafa6254efe None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:03.897622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-24c99ccb-4045-49ba-9ed1-aeafa6254efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:03.913984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:03.974465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 17/33] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:03.975456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c99ccb-4045-49ba-9ed1-aeafa6254efe None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:04.082112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 17/34] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 22 10:45:03 2022] POST /identity/v3/roles => generated 219 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:04.084032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:04.146355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c99ccb-4045-49ba-9ed1-aeafa6254efe None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:04.184866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b68a6b-763b-4027-b611-92de7bdd6814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:04.185860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68a6b-763b-4027-b611-92de7bdd6814 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.186366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68a6b-763b-4027-b611-92de7bdd6814 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.186856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68a6b-763b-4027-b611-92de7bdd6814 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.187960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b68a6b-763b-4027-b611-92de7bdd6814 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:04.189614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b68a6b-763b-4027-b611-92de7bdd6814 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:04.244607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1211-4ddd-4c60-9ba3-1cd008fc0579 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:04.273819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:04.294107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5cd3f-e56a-4717-a1c5-d87a8d1c7a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:04.294803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5cd3f-e56a-4717-a1c5-d87a8d1c7a6e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.295090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5cd3f-e56a-4717-a1c5-d87a8d1c7a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.295370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5cd3f-e56a-4717-a1c5-d87a8d1c7a6e None admin] PATH_INFO: `/v3/users/5f27905b35c14e3c9541d9819a2b2185` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.296669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5cd3f-e56a-4717-a1c5-d87a8d1c7a6e None admin] RBAC: Authorizing `identity:get_user(user_id=5f27905b35c14e3c9541d9819a2b2185)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:04.298447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5cd3f-e56a-4717-a1c5-d87a8d1c7a6e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:04.300037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 18/35] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:04 2022] GET /identity/v3/users/5f27905b35c14e3c9541d9819a2b2185 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:04.301518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0dae-8068-4885-889b-77f42b2ab809 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.301810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0dae-8068-4885-889b-77f42b2ab809 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.302142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e0dae-8068-4885-889b-77f42b2ab809 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.303724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 19/36] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:04.305821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e0dae-8068-4885-889b-77f42b2ab809 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:04.426965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e0dae-8068-4885-889b-77f42b2ab809 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:04.433508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 18/37] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:45:03 2022] POST /identity/v3/projects => generated 301 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:04.435348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251ebd3-d7f4-410a-b14b-499e5a13417c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.435698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251ebd3-d7f4-410a-b14b-499e5a13417c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.436068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251ebd3-d7f4-410a-b14b-499e5a13417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.449240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:04.465850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-5251ebd3-d7f4-410a-b14b-499e5a13417c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:04.467466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5251ebd3-d7f4-410a-b14b-499e5a13417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:04.473886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:04.474819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.475214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.475493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.477007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:04.478522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:04.482362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 20/38] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:04.482362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:04.556734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:04.563756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 19/39] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:04.564877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea73c55-154c-41fc-aaf1-cf8a72af5dae None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.565171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea73c55-154c-41fc-aaf1-cf8a72af5dae None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.565446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea73c55-154c-41fc-aaf1-cf8a72af5dae None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.567506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 20/40] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:04.571122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea73c55-154c-41fc-aaf1-cf8a72af5dae None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:04.620153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf57c1-d852-4bb1-a92b-fcc3650c1c6f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:04.666174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a02d9-9aca-4941-a4b3-b1036f1d6832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:04.666954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a02d9-9aca-4941-a4b3-b1036f1d6832 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.667245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a02d9-9aca-4941-a4b3-b1036f1d6832 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.667524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a02d9-9aca-4941-a4b3-b1036f1d6832 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.668186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a02d9-9aca-4941-a4b3-b1036f1d6832 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:04.669578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a02d9-9aca-4941-a4b3-b1036f1d6832 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:04.701009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea73c55-154c-41fc-aaf1-cf8a72af5dae None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:04.731956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:04.738203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:04.738901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.739177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.739456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] PATH_INFO: `/v3/domains/default/users/5f27905b35c14e3c9541d9819a2b2185/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.742437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5f27905b35c14e3c9541d9819a2b2185, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:04.743275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 21/41] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 22 10:45:04 2022] POST /identity/v3/roles => generated 225 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:04.744659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680660c-cb1b-4fd1-8739-0dec0037d60d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:04.745139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680660c-cb1b-4fd1-8739-0dec0037d60d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:04.745556 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680660c-cb1b-4fd1-8739-0dec0037d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:04.747524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:04.769166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9680660c-cb1b-4fd1-8739-0dec0037d60d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:04.771235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9680660c-cb1b-4fd1-8739-0dec0037d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:04.857798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 21/42] 10.222.0.48 () {58 vars in 1361 bytes} [Mon Aug 22 10:45:04 2022] PUT /identity/v3/domains/default/users/5f27905b35c14e3c9541d9819a2b2185/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:04.861781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:04.861951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 22/43] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:05.002621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325e50c-3db5-4996-81cf-bf0bfaabbe78 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:05.039499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:05.040121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:05.040403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:05.040678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:05.041343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:05.042838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:05.095390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 22/44] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 22 10:45:04 2022] POST /identity/v3/roles => generated 223 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:05.128197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680660c-cb1b-4fd1-8739-0dec0037d60d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:05.298351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:05.350619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680660c-cb1b-4fd1-8739-0dec0037d60d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:05.384063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2653b-51be-483d-99c5-da211ace8716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:05.384710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2653b-51be-483d-99c5-da211ace8716 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:05.384993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2653b-51be-483d-99c5-da211ace8716 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:05.385270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2653b-51be-483d-99c5-da211ace8716 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:05.386418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2653b-51be-483d-99c5-da211ace8716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:05.387873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2653b-51be-483d-99c5-da211ace8716 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:05.389455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 23/45] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:05.394964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:05.446867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:05.493431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0df90-36ba-45ec-b67a-94178afd0805 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:05.527591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a09ff-bd69-4265-82e9-4f611d986760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:05.528288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a09ff-bd69-4265-82e9-4f611d986760 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:05.528574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a09ff-bd69-4265-82e9-4f611d986760 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:05.528854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a09ff-bd69-4265-82e9-4f611d986760 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:05.529511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a09ff-bd69-4265-82e9-4f611d986760 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:05.531066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a09ff-bd69-4265-82e9-4f611d986760 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:05.650494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 23/46] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:45:05 2022] POST /identity/v3/projects => generated 312 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:06.805058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765dbaa-a91b-4aea-86d2-f3868d2fdeb4 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:06.805596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765dbaa-a91b-4aea-86d2-f3868d2fdeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:06.805677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765dbaa-a91b-4aea-86d2-f3868d2fdeb4 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:06.807015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 24/47] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:06.812273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766ae62-aab5-4dc1-bcae-655b9b91d6ed None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:06.812656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766ae62-aab5-4dc1-bcae-655b9b91d6ed None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:06.812940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766ae62-aab5-4dc1-bcae-655b9b91d6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:06.830819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7766ae62-aab5-4dc1-bcae-655b9b91d6ed None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:06.831997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7766ae62-aab5-4dc1-bcae-655b9b91d6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:06.901639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 24/48] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:07.173015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765dbaa-a91b-4aea-86d2-f3868d2fdeb4 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:07.274845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765dbaa-a91b-4aea-86d2-f3868d2fdeb4 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:07.292466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:07.309159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:07.312151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:07.312151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:07.312151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:07.321335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:07.326434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:07.327236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:07.328084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 25/49] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:07.333293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766ae62-aab5-4dc1-bcae-655b9b91d6ed None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:07.367656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:07.433292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766ae62-aab5-4dc1-bcae-655b9b91d6ed None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:07.466476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f07f9-48ee-4cbd-a80f-6687bf5641d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:07.467088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f07f9-48ee-4cbd-a80f-6687bf5641d0 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:07.467373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f07f9-48ee-4cbd-a80f-6687bf5641d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:07.467650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f07f9-48ee-4cbd-a80f-6687bf5641d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:07.468323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f07f9-48ee-4cbd-a80f-6687bf5641d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:07.468630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f07f9-48ee-4cbd-a80f-6687bf5641d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:07.470022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f07f9-48ee-4cbd-a80f-6687bf5641d0 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:07.478181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 25/50] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:07.483703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:07.583369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b9b7-5a60-49f7-bf35-309d976cd8a4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:07.594399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dd3b1-125d-4728-831b-8f6b1b9394c4 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:07.594691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dd3b1-125d-4728-831b-8f6b1b9394c4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:07.594970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dd3b1-125d-4728-831b-8f6b1b9394c4 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:07.596442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 26/51] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:45:07.601761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22936216-09c8-408d-8248-95bb9d310939 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:07.602051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22936216-09c8-408d-8248-95bb9d310939 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:07.602363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22936216-09c8-408d-8248-95bb9d310939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:07.618718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:07.619358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:07.619639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:07.619912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] PATH_INFO: `/v3/users/5f27905b35c14e3c9541d9819a2b2185` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:07.620118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-22936216-09c8-408d-8248-95bb9d310939 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:07.621014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] RBAC: Authorizing `identity:get_user(user_id=5f27905b35c14e3c9541d9819a2b2185)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:07.621321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-22936216-09c8-408d-8248-95bb9d310939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:07.622761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:07.624327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 26/52] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:07 2022] GET /identity/v3/users/5f27905b35c14e3c9541d9819a2b2185 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:07.629683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:07.674911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:07.688428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 27/53] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:07.728233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5890b-bcea-48de-be46-e09245850e96 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:07.762468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:07.763098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:07.763379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:07.763653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:07.764748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:07.766197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:07.767812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 27/54] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:07.773696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22936216-09c8-408d-8248-95bb9d310939 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:07.800276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:07.873003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22936216-09c8-408d-8248-95bb9d310939 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:07.954893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:08.012348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:08.012954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:08.013233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:08.013508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:08.014250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.domain.id=default, user.id=5f27905b35c14e3c9541d9819a2b2185) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:08.014558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:08.015987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:08.024372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 28/55] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 22 10:45:07 2022] GET /identity/v3/role_assignments?user.id=5f27905b35c14e3c9541d9819a2b2185&scope.domain.id=default&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 530 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:08.064012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9949a-c0d6-4f62-8274-c1c9b3ce05a9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:08.079681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:08.101357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925d02c-eeb8-4d40-86b4-d27e1d497c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:08.102010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d02c-eeb8-4d40-86b4-d27e1d497c60 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:08.102324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d02c-eeb8-4d40-86b4-d27e1d497c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:08.102597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925d02c-eeb8-4d40-86b4-d27e1d497c60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:08.103726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925d02c-eeb8-4d40-86b4-d27e1d497c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:08.105216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925d02c-eeb8-4d40-86b4-d27e1d497c60 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:08.106844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 28/56] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:08.112614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:08.206509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:08.217302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62393a11-6501-4b43-ad8b-74c3884c76a1 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:08.251575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90dd1e-d600-4d02-b78d-33eaa2c2421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:08.252188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90dd1e-d600-4d02-b78d-33eaa2c2421d None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:08.252470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90dd1e-d600-4d02-b78d-33eaa2c2421d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:08.252747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90dd1e-d600-4d02-b78d-33eaa2c2421d None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:08.253397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90dd1e-d600-4d02-b78d-33eaa2c2421d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:08.254880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90dd1e-d600-4d02-b78d-33eaa2c2421d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:08.347241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 29/57] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 22 10:45:08 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:10.250055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee776a-5b51-4811-a64d-903e2ee3a506 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:10.250621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee776a-5b51-4811-a64d-903e2ee3a506 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:10.250621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee776a-5b51-4811-a64d-903e2ee3a506 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:10.251989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 29/58] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:10.257279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a6dfc-c71d-45c0-b2a4-e76b275f826f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:10.257669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a6dfc-c71d-45c0-b2a4-e76b275f826f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:10.257911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a6dfc-c71d-45c0-b2a4-e76b275f826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:10.276254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-db9a6dfc-c71d-45c0-b2a4-e76b275f826f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:10.277448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-db9a6dfc-c71d-45c0-b2a4-e76b275f826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:10.340689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 30/59] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:10.607199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee776a-5b51-4811-a64d-903e2ee3a506 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:10.708202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee776a-5b51-4811-a64d-903e2ee3a506 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:10.742765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69e562-35d8-4d8a-8091-6d21994ed44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:10.743397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69e562-35d8-4d8a-8091-6d21994ed44a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:10.743677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69e562-35d8-4d8a-8091-6d21994ed44a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:10.743949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69e562-35d8-4d8a-8091-6d21994ed44a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:10.745046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69e562-35d8-4d8a-8091-6d21994ed44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:10.746514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69e562-35d8-4d8a-8091-6d21994ed44a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:10.748306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 30/60] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:10.753803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a6dfc-c71d-45c0-b2a4-e76b275f826f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:10.820624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:10.848232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a6dfc-c71d-45c0-b2a4-e76b275f826f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:10.883407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cb5dc-8561-4c5d-bd6a-5746cb22d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:10.884005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb5dc-8561-4c5d-bd6a-5746cb22d082 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:10.884284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb5dc-8561-4c5d-bd6a-5746cb22d082 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:10.884556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb5dc-8561-4c5d-bd6a-5746cb22d082 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:10.885191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cb5dc-8561-4c5d-bd6a-5746cb22d082 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:10.886667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cb5dc-8561-4c5d-bd6a-5746cb22d082 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:10.957241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 31/61] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:45:10 2022] POST /identity/v3/users => generated 275 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:13.921834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7355e7-57bd-4cf5-a826-93de4703f734 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:13.922806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7355e7-57bd-4cf5-a826-93de4703f734 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:13.923296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7355e7-57bd-4cf5-a826-93de4703f734 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:13.924881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 31/62] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:13.930652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310dadc-31ed-44bd-b5c5-06c3689bb0bb None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:13.931533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310dadc-31ed-44bd-b5c5-06c3689bb0bb None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:13.932001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310dadc-31ed-44bd-b5c5-06c3689bb0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:13.950913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e310dadc-31ed-44bd-b5c5-06c3689bb0bb None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:13.952431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e310dadc-31ed-44bd-b5c5-06c3689bb0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:14.027628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 32/63] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:14.229606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb4dc-526b-41e4-9460-a85196919459 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.230309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb4dc-526b-41e4-9460-a85196919459 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.230802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb4dc-526b-41e4-9460-a85196919459 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:14.232356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 32/64] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:14.240755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79cad-c759-4bdb-a067-ca92ea08c1d8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.241783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79cad-c759-4bdb-a067-ca92ea08c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.242417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79cad-c759-4bdb-a067-ca92ea08c1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:14.264626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-15c79cad-c759-4bdb-a067-ca92ea08c1d8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:14.266216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-15c79cad-c759-4bdb-a067-ca92ea08c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:14.277168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ad07-58a7-4538-a9ee-a62027e97a63 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.277780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ad07-58a7-4538-a9ee-a62027e97a63 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.278287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ad07-58a7-4538-a9ee-a62027e97a63 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:14.279904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 33/65] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:14.289677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267877b-1a5a-45a8-a2e2-d23c9b573073 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.290292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267877b-1a5a-45a8-a2e2-d23c9b573073 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.290784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267877b-1a5a-45a8-a2e2-d23c9b573073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:14.316535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a267877b-1a5a-45a8-a2e2-d23c9b573073 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:14.318804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a267877b-1a5a-45a8-a2e2-d23c9b573073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:14.391218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:14.392669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:14.410476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 33/66] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:14.412183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79cad-c759-4bdb-a067-ca92ea08c1d8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:14.435658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 34/67] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:14.516882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79cad-c759-4bdb-a067-ca92ea08c1d8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:14.553275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:14.554196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.554706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.555148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:14.566780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:14.573843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:14.574703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:14.575453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:14.577297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 34/68] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:14.583900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267877b-1a5a-45a8-a2e2-d23c9b573073 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:14.747946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267877b-1a5a-45a8-a2e2-d23c9b573073 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:14.778644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:14.814444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:14.815075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.815357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.815632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:14.816310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:14.816613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:14.819312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:14.830001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 35/69] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:14.834354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:14.846035 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:14.885221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:14.904857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8551e-4dcc-4e2c-96ba-19d965522afb None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:14.939974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:14.940621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.940876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.941145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:14.946726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81644d-8a08-47d3-bbb6-80eedc9107f2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:14.952607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:14.957772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:14.958730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:14.959475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 35/70] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:14.960083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:14.986976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887eaf-1bb6-4516-b5ef-7e7ce1eaa131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:14.987724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887eaf-1bb6-4516-b5ef-7e7ce1eaa131 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:14.988004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887eaf-1bb6-4516-b5ef-7e7ce1eaa131 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:14.988274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887eaf-1bb6-4516-b5ef-7e7ce1eaa131 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.000811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20887eaf-1bb6-4516-b5ef-7e7ce1eaa131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-20887eaf-1bb6-4516-b5ef-7e7ce1eaa131 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:15.007367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:15.008289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:15.009035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:15.014407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 36/71] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:15.014407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a50b9-c3f2-4585-87d2-93b1dbefd784 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.014407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a50b9-c3f2-4585-87d2-93b1dbefd784 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.014407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a50b9-c3f2-4585-87d2-93b1dbefd784 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.014407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 37/72] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:15.015473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a50b9-c3f2-4585-87d2-93b1dbefd784 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.065759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36e4ab-047f-455d-9d98-6eb046d8fcee None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.074826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.099248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.099873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.100463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.100928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.116709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.118164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.119972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 36/73] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:14 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:15.120627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.122180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a50b9-c3f2-4585-87d2-93b1dbefd784 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.123761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.157259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980e06a-36f4-4739-8ce6-98e7698edc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.157864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980e06a-36f4-4739-8ce6-98e7698edc40 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.158148 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980e06a-36f4-4739-8ce6-98e7698edc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.158452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980e06a-36f4-4739-8ce6-98e7698edc40 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.159114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980e06a-36f4-4739-8ce6-98e7698edc40 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:15.159419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980e06a-36f4-4739-8ce6-98e7698edc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.160874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980e06a-36f4-4739-8ce6-98e7698edc40 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.170335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 38/74] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:15.171493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff98c-03fd-4824-b601-ae60ba7f3862 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.171776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff98c-03fd-4824-b601-ae60ba7f3862 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.172052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ff98c-03fd-4824-b601-ae60ba7f3862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.189641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f53ff98c-03fd-4824-b601-ae60ba7f3862 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:15.190854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f53ff98c-03fd-4824-b601-ae60ba7f3862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:15.221616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10710aa-2b5e-41fc-900f-ae2bdaa5277b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.256406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.257044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.257328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.257517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 39/75] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:15.257603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.258289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:15.258597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.258856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664aca4-d11e-437f-a15c-5a8efaf52b25 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.259146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664aca4-d11e-437f-a15c-5a8efaf52b25 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.259419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664aca4-d11e-437f-a15c-5a8efaf52b25 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.259980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.260639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 40/76] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:15.261327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664aca4-d11e-437f-a15c-5a8efaf52b25 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.275325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 37/77] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:15.276413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.305217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.365066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.373208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664aca4-d11e-437f-a15c-5a8efaf52b25 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.381856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403bf45-5087-43fb-aa01-085ffd41e8dc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.409438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.410064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.410446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.410710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.417309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037b202-bab5-49be-90a0-cf51562f2a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.417941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037b202-bab5-49be-90a0-cf51562f2a40 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.418262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037b202-bab5-49be-90a0-cf51562f2a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.418523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037b202-bab5-49be-90a0-cf51562f2a40 None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.419661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037b202-bab5-49be-90a0-cf51562f2a40 None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.421094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037b202-bab5-49be-90a0-cf51562f2a40 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.422891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 38/78] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:15.423861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287718c-445a-419b-b020-a612a8695236 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.424145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287718c-445a-419b-b020-a612a8695236 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.424418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287718c-445a-419b-b020-a612a8695236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.444642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7287718c-445a-419b-b020-a612a8695236 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:15.445846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7287718c-445a-419b-b020-a612a8695236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:15.482824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.484273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.515950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 39/79] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:15.532401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287718c-445a-419b-b020-a612a8695236 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.561084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.586946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.599425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 41/80] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:15.599425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.647532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287718c-445a-419b-b020-a612a8695236 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.681716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.682390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.682674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.682950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] PATH_INFO: `/v3/users/5f27905b35c14e3c9541d9819a2b2185` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.684105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] RBAC: Authorizing `identity:get_user(user_id=5f27905b35c14e3c9541d9819a2b2185)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.685560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.687171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 40/81] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/users/5f27905b35c14e3c9541d9819a2b2185 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:15.687934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.715305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18da133-d915-4c97-9470-adec5d65774b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.757767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.758791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.759372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.759649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.782801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.790120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f8175-062e-42ed-9676-cf756feff021 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.822713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:15.824161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.824538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.824818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.825088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.826184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.827650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.829208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 41/82] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:15.829208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.831338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.833378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.933106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c95bc-28a2-4972-98f2-2e0407b385ca None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:15.935908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 42/83] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:15.936827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:15.966841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:15.967477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:15.967759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:15.968036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:15.969161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:15.970640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:15.972508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 42/84] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:15.973023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.005348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.029599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.036786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2e0d7-57e2-4e35-a700-236ed66ed390 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.069926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.070587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.071064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.071510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.076894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21663a47-b5a2-41a6-9055-45e162a26eb4 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.110493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.111124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.111408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.111687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.122520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:16.127167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:16.127988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:16.128718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 43/85] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:16.130391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.142782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.144252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.218295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.227659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab341ce-77a8-4e29-b72e-4491bbb41fdc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.234813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.242544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 43/86] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:16.243358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.261686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.263275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.263275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.263275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.263601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.264954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.339835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 44/87] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:45:16 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:16.340752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.344226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9a8b3-084e-45d8-8bd8-703a4deccc35 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.385709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.386418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.386701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.387028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.388273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.389814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.397187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 44/88] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:16.397187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.461369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd0bfc-00e0-4c3e-a98e-59f4ccb1b667 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.466208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.502322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.502845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.503131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.503410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.504160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=61475891ae6e4dce9c537af85750743e) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:16.504474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.506565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.512513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 45/89] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/role_assignments?user.id=61475891ae6e4dce9c537af85750743e&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:16.515808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.515942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4bbe5-d03b-424a-bdae-c183ffc2fee4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.528486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.624518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.625164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.625439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.625709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.627251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:16.627550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.629004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.640210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 45/90] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:16.640866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.668431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf36fe-84dc-44ca-b95d-1b9b8c86aad6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.714265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.714948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.715231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.715509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.717310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.718813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.720620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 46/91] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:16.721398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.783352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb326a9-992b-45b8-b4b2-617326a9be59 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.816707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.827528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:16.828998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.829791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.830247 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] RBAC: Adding query filter params (role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=61475891ae6e4dce9c537af85750743e) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:16.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.833712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.840054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 46/92] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/role_assignments?user.id=61475891ae6e4dce9c537af85750743e&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:16.841078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:16.852666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48255477-f6c9-4284-8c27-588bc041ed66 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:16.899524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:16.900175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:16.900457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:16.900735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:16.901901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:16.904084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:16.906259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 47/93] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:16.915675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:17.068176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d1f4-642b-48fb-ac41-b2b149141b8f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:17.108533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:17.109148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:17.109427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:17.109699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:17.127876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:17.147580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c2dd5-2eba-4e4a-b6a3-e6d5192df6ec None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:17.190219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:17.192293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:17.220744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:17.221467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:17.221758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:17.222037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:17.222809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=5f27905b35c14e3c9541d9819a2b2185) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:17.223225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:17.225471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:17.238533 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 47/94] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/role_assignments?user.id=5f27905b35c14e3c9541d9819a2b2185&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:17.307720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 48/95] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:17.313755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:17.401816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:17.460125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76c07b-09da-40d7-ad25-7aedb9d08b1a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:17.508910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82cfd5-edeb-4b76-b0d1-cf89a4540160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:17.509568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cfd5-edeb-4b76-b0d1-cf89a4540160 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:17.509852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cfd5-edeb-4b76-b0d1-cf89a4540160 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:17.510133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82cfd5-edeb-4b76-b0d1-cf89a4540160 None admin] PATH_INFO: `/v3/projects/837138e03cf64b7c93aaa0e19db792e5` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:17.511329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82cfd5-edeb-4b76-b0d1-cf89a4540160 None admin] RBAC: Authorizing `identity:get_project(project_id=837138e03cf64b7c93aaa0e19db792e5)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:17.512792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82cfd5-edeb-4b76-b0d1-cf89a4540160 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:17.514516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 48/96] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:17 2022] GET /identity/v3/projects/837138e03cf64b7c93aaa0e19db792e5 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:17.520537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:17.578607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:17.638990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f1791-bdc1-4339-b52a-92ac1c499c97 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:17.674645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec180e-1879-415f-b7d6-1cb10a1dd206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:17.675174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec180e-1879-415f-b7d6-1cb10a1dd206 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:17.675454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec180e-1879-415f-b7d6-1cb10a1dd206 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:17.675730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec180e-1879-415f-b7d6-1cb10a1dd206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:17.676502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec180e-1879-415f-b7d6-1cb10a1dd206 None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=837138e03cf64b7c93aaa0e19db792e5, user.id=61475891ae6e4dce9c537af85750743e) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:17.676816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec180e-1879-415f-b7d6-1cb10a1dd206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:17.678269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec180e-1879-415f-b7d6-1cb10a1dd206 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:17.683671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 49/97] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:17 2022] GET /identity/v3/role_assignments?user.id=61475891ae6e4dce9c537af85750743e&scope.project.id=837138e03cf64b7c93aaa0e19db792e5&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:19.096363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080bb30-35b1-43ef-9449-c3eab9f50290 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.096860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080bb30-35b1-43ef-9449-c3eab9f50290 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.096964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080bb30-35b1-43ef-9449-c3eab9f50290 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.099005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 49/98] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:19.104310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d375a0-223a-4892-b00b-b809f4c7bfee None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.104716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d375a0-223a-4892-b00b-b809f4c7bfee None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.104883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d375a0-223a-4892-b00b-b809f4c7bfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.126475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-98d375a0-223a-4892-b00b-b809f4c7bfee None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:19.127613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-98d375a0-223a-4892-b00b-b809f4c7bfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:19.214629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 50/99] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:19.232927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae117668-cdd1-4f88-bbbf-d45be597051c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.233217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae117668-cdd1-4f88-bbbf-d45be597051c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.233482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae117668-cdd1-4f88-bbbf-d45be597051c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.234878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 50/100] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:19.244956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546e6a-17f3-4c2c-acd8-fb2d05cdaa75 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.245247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546e6a-17f3-4c2c-acd8-fb2d05cdaa75 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.245521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2546e6a-17f3-4c2c-acd8-fb2d05cdaa75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.267231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b2546e6a-17f3-4c2c-acd8-fb2d05cdaa75 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:19.268484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b2546e6a-17f3-4c2c-acd8-fb2d05cdaa75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:19.315645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:19.357339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 51/101] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:19.417355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c9149-7110-4675-b95f-1671c2f75a7b None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.417641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c9149-7110-4675-b95f-1671c2f75a7b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.417836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c9149-7110-4675-b95f-1671c2f75a7b None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.419353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 51/102] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:19.424639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c24dd3-80ae-4fb9-83a8-efcbdb0f82be None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.425149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c24dd3-80ae-4fb9-83a8-efcbdb0f82be None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.425630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c24dd3-80ae-4fb9-83a8-efcbdb0f82be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.460645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d5c24dd3-80ae-4fb9-83a8-efcbdb0f82be None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:19.462172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d5c24dd3-80ae-4fb9-83a8-efcbdb0f82be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:19.549449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 52/103] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:19.636511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c9149-7110-4675-b95f-1671c2f75a7b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:19.699934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ab7c1-97e6-4d40-b6c8-2c96bedf7406 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.700526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ab7c1-97e6-4d40-b6c8-2c96bedf7406 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.701025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ab7c1-97e6-4d40-b6c8-2c96bedf7406 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.702903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 53/104] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:19.707475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b11ea-c132-4e03-8dc1-1ddba6c2f3c5 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.708078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b11ea-c132-4e03-8dc1-1ddba6c2f3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.708559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b11ea-c132-4e03-8dc1-1ddba6c2f3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.735529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-0d0b11ea-c132-4e03-8dc1-1ddba6c2f3c5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:19.737040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0d0b11ea-c132-4e03-8dc1-1ddba6c2f3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:19.792519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c9149-7110-4675-b95f-1671c2f75a7b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:19.830506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:19.865232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 54/105] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:19.874252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:19.874252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:19.874252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:19.874252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:19.878883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:19.880884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:19.882518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 52/106] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:19.887288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b11ea-c132-4e03-8dc1-1ddba6c2f3c5 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:19.888009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:19.962216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:20.104680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b11ea-c132-4e03-8dc1-1ddba6c2f3c5 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.111855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5be79b-e76a-41df-9c86-216778ce2dca None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.152796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.153866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.155468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.155952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.157498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:20.159741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:20.171984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.172859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.177876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.178453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.197622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:20.204330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:20.205186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:20.205928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:20.208661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 55/107] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:20.209757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:20.291278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 53/108] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:45:19 2022] POST /identity/v3/projects => generated 302 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:45:20.292510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:20.320273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:20.336750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4cf9f-fbbc-45c6-a467-0f9bb148a66a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.383192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:20.384669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.385328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.385615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.385891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.414272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:20.420250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:20.421148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:20.421960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 56/109] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:20.423703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:20.490885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff255c6-dcbb-4565-8b55-42c5320b18c6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.536934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.537574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.537860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.538137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.538862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:20.539225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:20.540638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:20.550324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 54/110] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:20.552052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:20.579011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36359ee-6870-4dc0-9ff7-4d52812679ea None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.626679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9ed77-81a4-4427-97d4-8087b9ad202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.627558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ed77-81a4-4427-97d4-8087b9ad202c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.627855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ed77-81a4-4427-97d4-8087b9ad202c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.628137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ed77-81a4-4427-97d4-8087b9ad202c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.644539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9ed77-81a4-4427-97d4-8087b9ad202c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a7c9ed77-81a4-4427-97d4-8087b9ad202c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:20.651404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:20.652263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:20.653011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 57/111] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:20.654729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a010-5cdd-4554-89cc-0b471bfc6f84 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.655013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a010-5cdd-4554-89cc-0b471bfc6f84 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.655319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a010-5cdd-4554-89cc-0b471bfc6f84 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.656691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 58/112] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:20.657283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a010-5cdd-4554-89cc-0b471bfc6f84 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:20.672755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:20.687312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04324e3f-5c84-4f18-85f3-f4641e109714 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.689271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:20.725750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774a9dd-8868-4378-8859-0e9240b59d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.726806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774a9dd-8868-4378-8859-0e9240b59d50 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.727092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774a9dd-8868-4378-8859-0e9240b59d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.727367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774a9dd-8868-4378-8859-0e9240b59d50 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.728049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774a9dd-8868-4378-8859-0e9240b59d50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:20.728353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774a9dd-8868-4378-8859-0e9240b59d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:20.730376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774a9dd-8868-4378-8859-0e9240b59d50 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:20.746990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 55/113] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:20.746990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de818e-a85b-4f4c-bcf1-79e130bfc4b9 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.746990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de818e-a85b-4f4c-bcf1-79e130bfc4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.746990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de818e-a85b-4f4c-bcf1-79e130bfc4b9 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.747707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 56/114] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:20.748133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de818e-a85b-4f4c-bcf1-79e130bfc4b9 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:20.782956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a010-5cdd-4554-89cc-0b471bfc6f84 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.818369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf9351-64c4-4938-bd91-821835a68004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.819044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf9351-64c4-4938-bd91-821835a68004 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.819331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf9351-64c4-4938-bd91-821835a68004 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.819608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf9351-64c4-4938-bd91-821835a68004 None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.820743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf9351-64c4-4938-bd91-821835a68004 None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:20.822177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf9351-64c4-4938-bd91-821835a68004 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:20.823932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 59/115] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:20 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:20.824875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7202645-2995-4d1f-b89b-3e72825b3234 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.825168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7202645-2995-4d1f-b89b-3e72825b3234 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.825427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7202645-2995-4d1f-b89b-3e72825b3234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.843139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f7202645-2995-4d1f-b89b-3e72825b3234 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:20.844324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f7202645-2995-4d1f-b89b-3e72825b3234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:20.862346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de818e-a85b-4f4c-bcf1-79e130bfc4b9 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:20.898588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:20.899262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.899699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.900032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.900729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:20.901027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:20.902549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:20.912914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 57/116] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:20.918042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:20.923541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 60/117] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:20.934269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5c792-5354-4839-ae88-b11d76b2ce6c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:20.934837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5c792-5354-4839-ae88-b11d76b2ce6c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:20.935396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5c792-5354-4839-ae88-b11d76b2ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:20.947718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:20.955306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b2f5c792-5354-4839-ae88-b11d76b2ce6c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:20.957334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b2f5c792-5354-4839-ae88-b11d76b2ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:20.960509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:21.027304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f3170-c847-4264-aed5-5817d4a94bc1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.036324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 61/118] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:21.037463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5c792-5354-4839-ae88-b11d76b2ce6c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.066431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.067702 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.067931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.068203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.069459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:21.071159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:21.073132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 58/119] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:20 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:21.073776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.181459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5c792-5354-4839-ae88-b11d76b2ce6c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.217576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.218555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.219262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.219762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.221154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:21.222828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:21.225551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 62/120] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:21 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:21.226551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.242583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:21.271507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:21.279291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84114-6dc4-497b-bece-a1c6ed9a9897 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.324915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.326045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.326356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.326694 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] PATH_INFO: `/v3/users/ee0d8f86da624bc8872f66e1f4d1f45a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.340863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac75fa-cc9c-4b8c-846a-efafa5b2cd40 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.345443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] RBAC: Authorizing `identity:get_user(user_id=ee0d8f86da624bc8872f66e1f4d1f45a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:21.347056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:21.350332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 59/121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:21 2022] GET /identity/v3/users/ee0d8f86da624bc8872f66e1f4d1f45a => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:21.351745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.383785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.384649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.385081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.385362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.386507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:21.387933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:21.389710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 63/122] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:21 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:21.390367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.467990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:21.472439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafd3d3-fea3-4226-a5a3-a33599759f9b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.496538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f2a6c-a584-4b71-a350-17d6efe8bf05 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.508764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.509438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.509719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.510053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd/users/61475891ae6e4dce9c537af85750743e/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.512014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=70b446c3c13d425d832af70478f512dd, user_id=61475891ae6e4dce9c537af85750743e, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:21.513467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:21.531980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.532615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.532925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.533210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.544194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:21.549687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:21.550827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:21.551597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 64/123] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:21.552193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.581239 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 60/124] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:21 2022] PUT /identity/v3/projects/70b446c3c13d425d832af70478f512dd/users/61475891ae6e4dce9c537af85750743e/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:21.581845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.603012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:21.688428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac2390-694a-4194-a9b6-c8b7be181e13 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.722034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ad9d3-3956-4be6-a0f2-71afcfc5aea8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.746127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.747327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.747742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.748244 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.753375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:21.772052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.773157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.773157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.773408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.791879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:21.798635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:21.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:21.800240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:21.803701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 61/125] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:21.804334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.837248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:21.838711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:21.898625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:21.938791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e3470-83f6-429b-b426-51db917b4a38 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:21.957635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 65/126] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:21.958484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:21.978630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:21.979616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:21.980098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:21.980667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd/users/61475891ae6e4dce9c537af85750743e/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:21.983327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=70b446c3c13d425d832af70478f512dd, user_id=61475891ae6e4dce9c537af85750743e, role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:21.985173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:21.998615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:22.071126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402607c-375c-49d2-8983-100829fc1789 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.093243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 62/127] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:21 2022] PUT /identity/v3/projects/70b446c3c13d425d832af70478f512dd/users/61475891ae6e4dce9c537af85750743e/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:22.094684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.138843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.139542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.139878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.140218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.141238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:22.141616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.143758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.162936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 66/128] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.163852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.266251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd660431-a887-4118-8eb7-02cff8e91cb5 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.304457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:22.310375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.310978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.320664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.320664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.320664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:22.320664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.320664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.327235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 63/129] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.328314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.359872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:22.368144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683241fa-525f-4b1b-a42b-6ac3857c88cc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.422702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.423573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.424014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.424436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.429802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.431564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.436654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 67/130] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:22 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.437612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.479790 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d6a88-3708-48c9-bac8-6b5f3468b03b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.524801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.526464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.526928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.527416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] PATH_INFO: `/v3/users/5f27905b35c14e3c9541d9819a2b2185` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.528723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] RBAC: Authorizing `identity:get_user(user_id=5f27905b35c14e3c9541d9819a2b2185)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.530937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.533372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 64/131] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:22 2022] GET /identity/v3/users/5f27905b35c14e3c9541d9819a2b2185 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.534677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.565220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fabc3-053d-4e1b-bb76-f42a694f2a5d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.610015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9456644-0205-4a53-81a5-cbd836856879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.610908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9456644-0205-4a53-81a5-cbd836856879 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.611199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9456644-0205-4a53-81a5-cbd836856879 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.611474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9456644-0205-4a53-81a5-cbd836856879 None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.612609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:22.613504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9456644-0205-4a53-81a5-cbd836856879 None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.615857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9456644-0205-4a53-81a5-cbd836856879 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.617533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 68/132] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:22 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.623130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c654da2-f7e1-4bac-a038-9e5ee53b5e0a None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.624531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c654da2-f7e1-4bac-a038-9e5ee53b5e0a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.625965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c654da2-f7e1-4bac-a038-9e5ee53b5e0a None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.627538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 69/133] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:22.628114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c654da2-f7e1-4bac-a038-9e5ee53b5e0a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.666856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:22.668310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bae2d-75aa-4ba9-9cba-aae61b668671 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.705910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.706939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.708152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.708642 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.709574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] RBAC: Adding query filter params (role.id=d2c9d0ae0229447bb2c52d38aa7e005c, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=ee0d8f86da624bc8872f66e1f4d1f45a) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:22.710361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.713346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.720946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 65/134] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:22 2022] GET /identity/v3/role_assignments?user.id=ee0d8f86da624bc8872f66e1f4d1f45a&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=d2c9d0ae0229447bb2c52d38aa7e005c => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.722059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.755396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c654da2-f7e1-4bac-a038-9e5ee53b5e0a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.795042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d057068-370b-47b4-97d8-1c18ac201766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.795686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057068-370b-47b4-97d8-1c18ac201766 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.795969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057068-370b-47b4-97d8-1c18ac201766 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.796243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057068-370b-47b4-97d8-1c18ac201766 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.797360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d057068-370b-47b4-97d8-1c18ac201766 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.799193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d057068-370b-47b4-97d8-1c18ac201766 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.800812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 70/135] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:22 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.802028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a88ee-dc9a-4a15-9aa3-86fa667f54e9 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.806306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a88ee-dc9a-4a15-9aa3-86fa667f54e9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.806306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a88ee-dc9a-4a15-9aa3-86fa667f54e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.821896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d63a88ee-dc9a-4a15-9aa3-86fa667f54e9 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:22.823152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d63a88ee-dc9a-4a15-9aa3-86fa667f54e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:22.843603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68771064-4559-41fe-891e-d810cd87a932 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:22.884489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:22.894444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:22.895723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:22.896294 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:22.896767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] PATH_INFO: `/v3/projects/837138e03cf64b7c93aaa0e19db792e5` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:22.904438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] RBAC: Authorizing `identity:get_project(project_id=837138e03cf64b7c93aaa0e19db792e5)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:22.906562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:22.912462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 66/136] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:22 2022] GET /identity/v3/projects/837138e03cf64b7c93aaa0e19db792e5 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:22.919189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.927146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 71/137] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:22.928180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a88ee-dc9a-4a15-9aa3-86fa667f54e9 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:22.980979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:23.048025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df234a-5196-41a0-b759-7572cc5ced16 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:23.086742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:23.087752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:23.088673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:23.089143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd/users/5f27905b35c14e3c9541d9819a2b2185/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:23.091308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] RBAC: Authorizing `identity:create_grant(project_id=70b446c3c13d425d832af70478f512dd, user_id=5f27905b35c14e3c9541d9819a2b2185, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:23.093299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:23.155697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a88ee-dc9a-4a15-9aa3-86fa667f54e9 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:23.170274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 67/138] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:22 2022] PUT /identity/v3/projects/70b446c3c13d425d832af70478f512dd/users/5f27905b35c14e3c9541d9819a2b2185/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:23.223220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:23.224569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:23.224856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:23.225134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] PATH_INFO: `/v3/projects/837138e03cf64b7c93aaa0e19db792e5/users/61475891ae6e4dce9c537af85750743e/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:23.229324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=837138e03cf64b7c93aaa0e19db792e5, user_id=61475891ae6e4dce9c537af85750743e, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:23.230831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:23.266087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:23.351795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 72/139] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:22 2022] PUT /identity/v3/projects/837138e03cf64b7c93aaa0e19db792e5/users/61475891ae6e4dce9c537af85750743e/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:23.359056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:23.414866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04f29d-ceb6-41fc-aed9-43d7f0940042 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:23.458055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ced774-1484-4af8-bf45-01fd32ffdcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:23.459146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ced774-1484-4af8-bf45-01fd32ffdcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:23.459625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ced774-1484-4af8-bf45-01fd32ffdcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:23.460088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ced774-1484-4af8-bf45-01fd32ffdcd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:23.467235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ced774-1484-4af8-bf45-01fd32ffdcd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:23.469150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ced774-1484-4af8-bf45-01fd32ffdcd0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:23.471443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 68/140] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:23.477417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:23.609763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:23.620501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf525f-933d-46ae-9bc6-6a7f0d4a75e2 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:23.664268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5cf7-1039-4a11-bff9-73bc56a710a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:23.664947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5cf7-1039-4a11-bff9-73bc56a710a7 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:23.665172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5cf7-1039-4a11-bff9-73bc56a710a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:23.665450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5cf7-1039-4a11-bff9-73bc56a710a7 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:23.666112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5cf7-1039-4a11-bff9-73bc56a710a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:23.667580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5cf7-1039-4a11-bff9-73bc56a710a7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:23.781051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 73/141] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:45:23 2022] POST /identity/v3/users => generated 283 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:23.855653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432ef7b-4a4c-4c5f-b375-665220c4ffef None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:23.855653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432ef7b-4a4c-4c5f-b375-665220c4ffef None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:23.855653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432ef7b-4a4c-4c5f-b375-665220c4ffef None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:23.855653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 69/142] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:45:23.876575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b044-99da-4160-b8ff-5cd1709b7ebc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:23.876888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b044-99da-4160-b8ff-5cd1709b7ebc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:23.877137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b044-99da-4160-b8ff-5cd1709b7ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:23.903974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-3a29b044-99da-4160-b8ff-5cd1709b7ebc None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:23.905211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3a29b044-99da-4160-b8ff-5cd1709b7ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:23.981442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 74/143] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:24.297419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432ef7b-4a4c-4c5f-b375-665220c4ffef None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:24.423756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432ef7b-4a4c-4c5f-b375-665220c4ffef None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:24.462616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:24.474894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:24.475830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:24.476394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:24.476901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:24.493393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:24.498823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:24.504671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:24.505563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 70/144] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:24.513269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b044-99da-4160-b8ff-5cd1709b7ebc None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:24.614714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:24.640286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b044-99da-4160-b8ff-5cd1709b7ebc None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:24.684680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:24.685307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:24.685589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:24.685865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:24.686587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] RBAC: Adding query filter params (name=member) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:24.686888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:24.688477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:24.697737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 75/145] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:24.708504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:24.835055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775ab96-b153-4cb0-bb39-c4e7751435e0 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:24.873167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:24.873753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:24.874043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:24.874358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:24.875513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:24.877010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:24.881015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 71/146] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:24 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:24.885131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:24.994041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60f226-50c8-457e-8dec-062df6f0cc8d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:25.029251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab237a7e-ef98-4411-89d1-0b08d01cfa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:25.029871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab237a7e-ef98-4411-89d1-0b08d01cfa24 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:25.030157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab237a7e-ef98-4411-89d1-0b08d01cfa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:25.030466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab237a7e-ef98-4411-89d1-0b08d01cfa24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:25.040379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:25.106379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab237a7e-ef98-4411-89d1-0b08d01cfa24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:25.111882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab237a7e-ef98-4411-89d1-0b08d01cfa24 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:25.230831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 76/147] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:25.236716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:25.250654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58e991-c0d0-4b70-8d78-790fa356b395 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:25.250935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58e991-c0d0-4b70-8d78-790fa356b395 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:25.251212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58e991-c0d0-4b70-8d78-790fa356b395 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:25.252712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 77/148] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:25.258804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd549ea-a7d7-42b7-805e-70e4f446b99c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:25.259139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd549ea-a7d7-42b7-805e-70e4f446b99c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:25.259416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd549ea-a7d7-42b7-805e-70e4f446b99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:25.298321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ddd549ea-a7d7-42b7-805e-70e4f446b99c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:25.299689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ddd549ea-a7d7-42b7-805e-70e4f446b99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:25.347453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:25.355360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:25.393991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 78/149] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:25.466319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c5fcf-eb67-4f69-9e22-c1fd17b6e3c9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:25.513891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:25.514485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:25.514767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:25.515040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:25.516187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:25.517641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:25.519593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 72/150] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:25 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:25.525945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd549ea-a7d7-42b7-805e-70e4f446b99c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:25.643945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd549ea-a7d7-42b7-805e-70e4f446b99c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:25.683148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8051c3-ff42-4744-bbd4-4ad03a307077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:25.683756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8051c3-ff42-4744-bbd4-4ad03a307077 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:25.684032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8051c3-ff42-4744-bbd4-4ad03a307077 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:25.684299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8051c3-ff42-4744-bbd4-4ad03a307077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:25.685033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8051c3-ff42-4744-bbd4-4ad03a307077 None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=61475891ae6e4dce9c537af85750743e) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:25.685335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8051c3-ff42-4744-bbd4-4ad03a307077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:25.686803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8051c3-ff42-4744-bbd4-4ad03a307077 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:25.695518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 79/151] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:25 2022] GET /identity/v3/role_assignments?user.id=61475891ae6e4dce9c537af85750743e&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:25.886219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:26.009084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f754a-1582-4c51-a1ed-a5a25e003535 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:26.028212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:26.045978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:26.046652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.046937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.047213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.060017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:26.064967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:26.065776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:26.066638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:26.068569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 73/152] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:26.073008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:26.073543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503c74f-bd27-4ad5-ab0a-0e73819cdef7 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.073940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503c74f-bd27-4ad5-ab0a-0e73819cdef7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.074262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503c74f-bd27-4ad5-ab0a-0e73819cdef7 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.075840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 80/153] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:26.084117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5834bf-db6f-4ef2-80a0-41856428bb24 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.084599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5834bf-db6f-4ef2-80a0-41856428bb24 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.085072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5834bf-db6f-4ef2-80a0-41856428bb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.121944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9c5834bf-db6f-4ef2-80a0-41856428bb24 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:26.123426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9c5834bf-db6f-4ef2-80a0-41856428bb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:26.180668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:26.187670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b9610-4ef5-40d3-ab94-5ace0c7aa543 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:26.226555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07e71c-52be-4d50-90d3-8bd210cc14dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:26.227177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07e71c-52be-4d50-90d3-8bd210cc14dc None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.227663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07e71c-52be-4d50-90d3-8bd210cc14dc None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.228081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07e71c-52be-4d50-90d3-8bd210cc14dc None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.228895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07e71c-52be-4d50-90d3-8bd210cc14dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:26.229343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07e71c-52be-4d50-90d3-8bd210cc14dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:26.231127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07e71c-52be-4d50-90d3-8bd210cc14dc None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:26.241734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 74/154] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:26.244949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27136a37-ec96-4245-99b5-983bed5dd608 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.245390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27136a37-ec96-4245-99b5-983bed5dd608 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.245913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27136a37-ec96-4245-99b5-983bed5dd608 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.247783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 75/155] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:26.251973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27136a37-ec96-4245-99b5-983bed5dd608 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:26.284564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 81/156] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:26.286334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdc1-a04e-4ee7-a681-af3997d996a0 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.286334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdc1-a04e-4ee7-a681-af3997d996a0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.286741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdc1-a04e-4ee7-a681-af3997d996a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.303078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:26.331301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4168bdc1-a04e-4ee7-a681-af3997d996a0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:26.333007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4168bdc1-a04e-4ee7-a681-af3997d996a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:26.371219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27136a37-ec96-4245-99b5-983bed5dd608 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:26.410109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:26.412090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.412518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.412970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.414462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:26.416243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:26.418183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 76/157] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:26 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:26.424289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:26.486125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 82/158] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:26.543684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023514b-03df-4124-9455-0852a1a1c393 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:26.555080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:26.581436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:26.582095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:26.583123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:26.583515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:26.658086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:26.659601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:26.783828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 77/159] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 10:45:26.790067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdc1-a04e-4ee7-a681-af3997d996a0 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:26.892421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:26.906568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:26.959542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:26.972088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdc1-a04e-4ee7-a681-af3997d996a0 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:27.046598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d53ba5-5d56-4173-9eff-e3247978366a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:27.055083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf217a-f624-499c-826e-bfa30727a99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:27.055744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf217a-f624-499c-826e-bfa30727a99e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.056029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf217a-f624-499c-826e-bfa30727a99e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.056308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf217a-f624-499c-826e-bfa30727a99e None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.057482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf217a-f624-499c-826e-bfa30727a99e None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:27.058952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf217a-f624-499c-826e-bfa30727a99e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:27.060607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 83/160] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:26 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:27.061728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22dddb-32ee-454f-98a0-1b4b594a4541 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.062006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22dddb-32ee-454f-98a0-1b4b594a4541 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.062299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22dddb-32ee-454f-98a0-1b4b594a4541 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.063634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 84/161] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:27.071179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfad1-7d68-4e92-ba90-75846511c0b4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.071464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfad1-7d68-4e92-ba90-75846511c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.071739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfad1-7d68-4e92-ba90-75846511c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.087869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:27.088519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.088801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.089077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.100958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:27.105966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:27.106848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:27.107659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:27.110356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 78/162] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:27.111184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:27.123232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-475dfad1-7d68-4e92-ba90-75846511c0b4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:27.124445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-475dfad1-7d68-4e92-ba90-75846511c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:27.186605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:27.222932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f509a-6d3b-41ef-8e9f-76dae6086a15 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:27.260641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:27.261261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.261588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.261917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.262691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] RBAC: Adding query filter params (role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=61475891ae6e4dce9c537af85750743e) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:27.263001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:27.264667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:27.272112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 79/163] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:27 2022] GET /identity/v3/role_assignments?user.id=61475891ae6e4dce9c537af85750743e&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:27.277318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:27.404675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 85/164] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:27.405315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfad1-7d68-4e92-ba90-75846511c0b4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:27.479498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd52e4-1b29-4429-9a3f-3ec500299b3b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:27.547444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:27.560855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:27.562637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.564040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.564780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.593792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:27.599010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:27.599874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:27.600622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:27.609488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 80/165] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:27.613079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:27.636169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:27.650638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfad1-7d68-4e92-ba90-75846511c0b4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:27.730617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:27.731249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.731622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.731900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.732629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:27.732938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:27.735183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:27.752963 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 86/166] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:27.759333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:27.826795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8b761-50ae-414d-8357-7bc2b716e947 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:27.903304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:27.903940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:27.904220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:27.904493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:27.905169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:27.905471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:27.906896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:27.919823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 81/167] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:27.920656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:27.963114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd12f8-cfba-4ecd-bbef-9b29425fd90a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.000955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.009821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.010665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.011090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.011506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] PATH_INFO: `/v3/users/ee0d8f86da624bc8872f66e1f4d1f45a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.013462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0d8f86da624bc8872f66e1f4d1f45a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:28.015441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:28.018338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 87/168] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:27 2022] GET /identity/v3/users/ee0d8f86da624bc8872f66e1f4d1f45a => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:28.019276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.048479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.056267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9430e3f-54ab-4046-9ff3-a4e8bb52b47e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.103353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2f024-7dbd-4726-a58d-8c7187215730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.104280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2f024-7dbd-4726-a58d-8c7187215730 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.104566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2f024-7dbd-4726-a58d-8c7187215730 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.104841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2f024-7dbd-4726-a58d-8c7187215730 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.119652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2f024-7dbd-4726-a58d-8c7187215730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-23f2f024-7dbd-4726-a58d-8c7187215730 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:28.128216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:28.129852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:28.130742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 82/169] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:28.131442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31ac7-198b-4fd6-8610-40ac7491e036 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.131505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31ac7-198b-4fd6-8610-40ac7491e036 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.132125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b31ac7-198b-4fd6-8610-40ac7491e036 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.133033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 83/170] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:28.135017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b31ac7-198b-4fd6-8610-40ac7491e036 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.163114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac7f183-24fd-4c2f-bd93-27deacff0db5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.208374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.209017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.209298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.209574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] PATH_INFO: `/v3/users/5f27905b35c14e3c9541d9819a2b2185` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.211032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] RBAC: Authorizing `identity:get_user(user_id=5f27905b35c14e3c9541d9819a2b2185)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:28.213131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:28.215077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 88/171] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:28 2022] GET /identity/v3/users/5f27905b35c14e3c9541d9819a2b2185 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:28.217202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.250687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b31ac7-198b-4fd6-8610-40ac7491e036 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.267149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.289394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593dd85-0d7f-4a1c-9e16-7b420e51399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.290439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593dd85-0d7f-4a1c-9e16-7b420e51399f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.291021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593dd85-0d7f-4a1c-9e16-7b420e51399f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.291490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593dd85-0d7f-4a1c-9e16-7b420e51399f None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.292937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593dd85-0d7f-4a1c-9e16-7b420e51399f None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:28.294857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593dd85-0d7f-4a1c-9e16-7b420e51399f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:28.297067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 84/172] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:28 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:28.298815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823117fa-d42c-44cd-9eb4-75f78600ddc0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.299654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823117fa-d42c-44cd-9eb4-75f78600ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.300180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823117fa-d42c-44cd-9eb4-75f78600ddc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.320565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-823117fa-d42c-44cd-9eb4-75f78600ddc0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:28.322246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-823117fa-d42c-44cd-9eb4-75f78600ddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:28.331929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.353418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c544e-f7d5-4ad3-9900-f8986236986e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.395983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.396616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.396882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.397223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.397824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] RBAC: Adding query filter params (name=member) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:28.398224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:28.399173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 85/173] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:28.399850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823117fa-d42c-44cd-9eb4-75f78600ddc0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.403885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:28.420241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 89/174] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:28.420241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.539048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823117fa-d42c-44cd-9eb4-75f78600ddc0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.539901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a29e1-3553-4c15-99d1-967b07bc275e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.558027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.577457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6927053-024d-4be9-af05-dc0144a967d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.578128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6927053-024d-4be9-af05-dc0144a967d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.578566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6927053-024d-4be9-af05-dc0144a967d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.578935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6927053-024d-4be9-af05-dc0144a967d3 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd/users/ee0d8f86da624bc8872f66e1f4d1f45a/roles/d2c9d0ae0229447bb2c52d38aa7e005c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.581207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6927053-024d-4be9-af05-dc0144a967d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=70b446c3c13d425d832af70478f512dd, user_id=ee0d8f86da624bc8872f66e1f4d1f45a, role_id=d2c9d0ae0229447bb2c52d38aa7e005c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:28.582730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6927053-024d-4be9-af05-dc0144a967d3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:28.586969 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207337b-747a-454a-abf3-64326ae524d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.587774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207337b-747a-454a-abf3-64326ae524d6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.588106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207337b-747a-454a-abf3-64326ae524d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.588377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207337b-747a-454a-abf3-64326ae524d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.651971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 90/175] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:28 2022] PUT /identity/v3/projects/70b446c3c13d425d832af70478f512dd/users/ee0d8f86da624bc8872f66e1f4d1f45a/roles/d2c9d0ae0229447bb2c52d38aa7e005c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:28.658685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b751c-9417-4597-8068-b47485cf2da2 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.658685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b751c-9417-4597-8068-b47485cf2da2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.658685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b751c-9417-4597-8068-b47485cf2da2 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.658685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 91/176] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:28.664372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b751c-9417-4597-8068-b47485cf2da2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.671517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207337b-747a-454a-abf3-64326ae524d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:28.673523 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207337b-747a-454a-abf3-64326ae524d6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:28.686707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.794068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b751c-9417-4597-8068-b47485cf2da2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:28.809112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 86/177] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:28.809112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655269d-3a39-48cc-a914-707e306535e7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.809112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655269d-3a39-48cc-a914-707e306535e7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.809112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655269d-3a39-48cc-a914-707e306535e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.829175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e655269d-3a39-48cc-a914-707e306535e7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:28.830396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e655269d-3a39-48cc-a914-707e306535e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:28.839708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.840319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:28.841328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.841620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.841897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] PATH_INFO: `/v3/users/61475891ae6e4dce9c537af85750743e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.843131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] RBAC: Authorizing `identity:get_user(user_id=61475891ae6e4dce9c537af85750743e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:28.844582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:28.846454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 92/178] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:28 2022] GET /identity/v3/users/61475891ae6e4dce9c537af85750743e => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:28.851707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.917110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 87/179] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:28.918542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77548e5f-5002-440f-87ef-1d8979af69cf None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:28.918892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77548e5f-5002-440f-87ef-1d8979af69cf None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:28.919275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77548e5f-5002-440f-87ef-1d8979af69cf None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:28.924764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 88/180] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:28.924764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77548e5f-5002-440f-87ef-1d8979af69cf None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:28.952702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:28.982345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc32ac-2c31-45af-969c-4d5e11cdff83 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.021292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.022084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.022539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.022914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.034039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:29.039838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:29.040857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:29.041810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:29.043771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 93/181] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:29.044799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.050600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77548e5f-5002-440f-87ef-1d8979af69cf None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.089957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f13bd-0156-41cb-8028-d4a69a3fa243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.090784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f13bd-0156-41cb-8028-d4a69a3fa243 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.091206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f13bd-0156-41cb-8028-d4a69a3fa243 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.092163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f13bd-0156-41cb-8028-d4a69a3fa243 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.092853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f13bd-0156-41cb-8028-d4a69a3fa243 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.094506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f13bd-0156-41cb-8028-d4a69a3fa243 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.101025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 89/182] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:28 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:29.101025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c529f-1d27-4fdf-b534-45739df7413a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.101025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c529f-1d27-4fdf-b534-45739df7413a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.101025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c529f-1d27-4fdf-b534-45739df7413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.118484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-bd7c529f-1d27-4fdf-b534-45739df7413a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:29.119863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bd7c529f-1d27-4fdf-b534-45739df7413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:29.123611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:29.167183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5770779-bf4d-40d2-8c93-43c45e4131d0 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.194276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 90/183] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:29.194969 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c529f-1d27-4fdf-b534-45739df7413a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.211871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.212469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.212746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.213016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.222589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:29.284216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.286358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.329295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c529f-1d27-4fdf-b534-45739df7413a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.374753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.375832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.376465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.376934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.378067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:29.378585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.380637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.391950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 91/184] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:29.392846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 94/185] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:29.393240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.393773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.396558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:29.438995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:29.503932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d79c0-a94a-413c-b7d8-2001346b1473 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.518319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c3ac5-c63d-48d9-b310-7119d7a8f495 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.540114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.540743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.541016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.541279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.552773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:29.555445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.556075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.556350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.556614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.557333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=5f27905b35c14e3c9541d9819a2b2185) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:29.557584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:29.558463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:29.563066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:29.563648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.564039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 95/186] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:29.564113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.564178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.570610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 92/187] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/role_assignments?user.id=5f27905b35c14e3c9541d9819a2b2185&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:29.571628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.686601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:29.692708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9603-0b49-475b-b65c-73f8e8933c23 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.702648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:29.706828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5213dc9-c8fe-40f8-b7a5-6dc28610058b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.744576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.746428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.746428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.746428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] PATH_INFO: `/v3/users/4528fd169d59409aada5eea70a06c1b1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.757750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.759143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.759143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.759143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] PATH_INFO: `/v3/projects/837138e03cf64b7c93aaa0e19db792e5` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.760117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] RBAC: Authorizing `identity:get_project(project_id=837138e03cf64b7c93aaa0e19db792e5)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.761963 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.763865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 93/188] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/projects/837138e03cf64b7c93aaa0e19db792e5 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:29.764668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.764917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] RBAC: Authorizing `identity:get_user(user_id=4528fd169d59409aada5eea70a06c1b1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.766382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.768593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 96/189] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/users/4528fd169d59409aada5eea70a06c1b1 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:29.770083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.891873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f9ef3-32cd-49c0-b62c-277edc5e99f3 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.893383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987efcd-d876-47a5-8e7c-6d88ee6f1bad None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:29.928284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.928447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:29.928906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.929189 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.929464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.929982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:29.930160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:29.930469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.930843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:29.931303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:29.931838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.932787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:29.934424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:29.935985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 94/190] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:29.939471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.940933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 97/191] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:29.942479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:29.943160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:29.962563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:30.043603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba13e1-6523-46a4-a173-692a0d1ee202 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:30.045985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e96e01d-3442-4ebb-9f4a-041e9afa3c5e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:30.078430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:30.079040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.079314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.079578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.080286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=837138e03cf64b7c93aaa0e19db792e5, user.id=61475891ae6e4dce9c537af85750743e) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:30.080385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:30.080596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:30.080991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.081276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.081536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.082661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:30.098268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 95/192] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/role_assignments?user.id=61475891ae6e4dce9c537af85750743e&scope.project.id=837138e03cf64b7c93aaa0e19db792e5&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:30.098268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:30.170837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:30.175549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:30.177330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:30.202616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:30.215998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650ab33-d8fe-43b9-9114-f5da87952321 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:30.258297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:30.259176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.259456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.259730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.261113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:30.262210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:30.301538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 98/193] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:30.301738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:30.349074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 96/194] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:45:30 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:30.350741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:30.438960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b227232-36a3-422e-a25f-c1a539e7805b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:30.464645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:30.487135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:30.487804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.489112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.489533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] PATH_INFO: `/v3/users/4528fd169d59409aada5eea70a06c1b1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.491787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] RBAC: Authorizing `identity:get_user(user_id=4528fd169d59409aada5eea70a06c1b1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:30.493858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:30.496867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 99/195] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:30 2022] GET /identity/v3/users/4528fd169d59409aada5eea70a06c1b1 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:30.502916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:30.508407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a67e7-1a8c-45a8-bacb-4b60e4e80498 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:30.540788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:30.556067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:30.556822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.557102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.557374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.560099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:30.562482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:30.565139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 97/196] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:30 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:30.571998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:30.633253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df5e7e-25dc-4096-9a65-de9799cea56c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:30.676972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:30.677598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.678300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.678300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.691933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9458a50-8395-4f1c-9cc8-64170971973d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:30.743239 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408edecc-2d19-4132-9277-3a6ed23fb7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:30.743868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edecc-2d19-4132-9277-3a6ed23fb7cf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.744148 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edecc-2d19-4132-9277-3a6ed23fb7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.744451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408edecc-2d19-4132-9277-3a6ed23fb7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.745171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edecc-2d19-4132-9277-3a6ed23fb7cf None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=4528fd169d59409aada5eea70a06c1b1) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:30.745474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edecc-2d19-4132-9277-3a6ed23fb7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:30.747225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408edecc-2d19-4132-9277-3a6ed23fb7cf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:30.749792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:30.753115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 98/197] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:30 2022] GET /identity/v3/role_assignments?user.id=4528fd169d59409aada5eea70a06c1b1&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:30.754836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65413795-6284-4722-bda4-af32bc3e0dc7 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.755118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65413795-6284-4722-bda4-af32bc3e0dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.755388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65413795-6284-4722-bda4-af32bc3e0dc7 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.756634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 99/198] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:30.762546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce295e04-2b9a-46be-bbc5-ade4a940a3be None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:30.766528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce295e04-2b9a-46be-bbc5-ade4a940a3be None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:30.766528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce295e04-2b9a-46be-bbc5-ade4a940a3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:30.766732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:30.767190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:30.786985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ce295e04-2b9a-46be-bbc5-ade4a940a3be None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:30.788562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ce295e04-2b9a-46be-bbc5-ade4a940a3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:30.871887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:30.888901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 100/199] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:30.951872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 100/200] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 10:45:30.958084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce295e04-2b9a-46be-bbc5-ade4a940a3be None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:31.080904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce295e04-2b9a-46be-bbc5-ade4a940a3be None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:31.125134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:31.125828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:31.126108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:31.126405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:31.127569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:31.129697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:31.131520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 101/201] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:30 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:31.137311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:31.199732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:31.230118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:31.243563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553882-ec03-4b4d-b75b-7afcda0ece4d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:31.285137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:31.286100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:31.286470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:31.286859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:31.287706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] RBAC: Adding query filter params (role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=4528fd169d59409aada5eea70a06c1b1) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:31.288269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:31.289722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:31.301513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 101/202] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:31 2022] GET /identity/v3/role_assignments?user.id=4528fd169d59409aada5eea70a06c1b1&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:31.333716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:31.411349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d941e-a70b-4eed-a250-3062d4cdac08 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:31.467396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:31.468037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:31.468311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:31.468580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:31.483936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:31.488954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:31.493276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:31.494027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 102/203] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:31.497017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:31.646026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaf350-8922-4d7a-b3d3-6bc9ecb58693 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:31.697358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:31.697987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:31.698306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:31.698591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:31.699265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:31.699567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:31.700943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:31.713785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 102/204] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:31.719231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:31.892116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e52c1-12a4-49a5-b70e-b3f83fec89dc None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:31.912640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:31.941202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:31.942487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:31.942758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:31.943029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] PATH_INFO: `/v3/users/ee0d8f86da624bc8872f66e1f4d1f45a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:31.946913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] RBAC: Authorizing `identity:get_user(user_id=ee0d8f86da624bc8872f66e1f4d1f45a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:31.948424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:31.950368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 103/205] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:31 2022] GET /identity/v3/users/ee0d8f86da624bc8872f66e1f4d1f45a => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:31.955824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:31.986916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:32.118348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e82f5e-4be4-4bd8-8b3d-5e2e7acbe864 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:32.167900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:32.168482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:32.168701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:32.168969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:32.261842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:32.263358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:32.280295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:32.390849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 103/206] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:32.396985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:32.508624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8bec6f-264c-404a-a9c7-d2d2442c583a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:32.550531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d709f4-3d43-439e-a707-dedb87ee8aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:32.551179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d709f4-3d43-439e-a707-dedb87ee8aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:32.551456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d709f4-3d43-439e-a707-dedb87ee8aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:32.551717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d709f4-3d43-439e-a707-dedb87ee8aa7 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:32.552851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d709f4-3d43-439e-a707-dedb87ee8aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:32.554330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d709f4-3d43-439e-a707-dedb87ee8aa7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:32.556276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 104/207] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:32 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:32.562189 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:32.686858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25907d21-f526-4543-b9c5-0aa9f76a78f9 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:32.710951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:32.745116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf12e6d-431f-4082-be05-1238b8be3836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:32.745797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf12e6d-431f-4082-be05-1238b8be3836 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:32.746261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf12e6d-431f-4082-be05-1238b8be3836 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:32.747105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf12e6d-431f-4082-be05-1238b8be3836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:32.747840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf12e6d-431f-4082-be05-1238b8be3836 None admin] RBAC: Adding query filter params (role.id=d2c9d0ae0229447bb2c52d38aa7e005c, scope.project.id=70b446c3c13d425d832af70478f512dd, user.id=ee0d8f86da624bc8872f66e1f4d1f45a) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:32.748147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf12e6d-431f-4082-be05-1238b8be3836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:32.749658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf12e6d-431f-4082-be05-1238b8be3836 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:32.759108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 104/208] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:32 2022] GET /identity/v3/role_assignments?user.id=ee0d8f86da624bc8872f66e1f4d1f45a&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=d2c9d0ae0229447bb2c52d38aa7e005c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:32.894582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00e1f-11a6-42dc-b8f3-c09a75e68489 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:32.895545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00e1f-11a6-42dc-b8f3-c09a75e68489 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:32.896020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a00e1f-11a6-42dc-b8f3-c09a75e68489 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:32.897671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 105/209] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:32.903588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b552a-31a3-4905-88ce-04da32626a1e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:32.904178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b552a-31a3-4905-88ce-04da32626a1e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:32.904649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b552a-31a3-4905-88ce-04da32626a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:32.931459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-823b552a-31a3-4905-88ce-04da32626a1e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:32.933040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-823b552a-31a3-4905-88ce-04da32626a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:32.998342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990d06f-925c-4b1c-afb8-2df90ac8fc0e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:32.998951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990d06f-925c-4b1c-afb8-2df90ac8fc0e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:32.999418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990d06f-925c-4b1c-afb8-2df90ac8fc0e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.000981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 106/210] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:45:33.006827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bc04a-49e6-452b-b45d-ede107220e3d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.007345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bc04a-49e6-452b-b45d-ede107220e3d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.007843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086bc04a-49e6-452b-b45d-ede107220e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.013645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 105/211] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:33.035567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-086bc04a-49e6-452b-b45d-ede107220e3d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:33.037131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-086bc04a-49e6-452b-b45d-ede107220e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:33.094863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:33.127046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 107/212] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:33.249908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3693d-9611-4135-bb72-339cfd331081 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.250421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3693d-9611-4135-bb72-339cfd331081 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.250492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3693d-9611-4135-bb72-339cfd331081 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.252124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 106/213] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:33.263888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67728091-1f52-45fc-beac-1154fc34bd2d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.264117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67728091-1f52-45fc-beac-1154fc34bd2d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.264431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67728091-1f52-45fc-beac-1154fc34bd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.285845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-67728091-1f52-45fc-beac-1154fc34bd2d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:33.287094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-67728091-1f52-45fc-beac-1154fc34bd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:33.334280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3693d-9611-4135-bb72-339cfd331081 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:33.367027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 108/214] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:33.427259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67728091-1f52-45fc-beac-1154fc34bd2d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:33.463345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3693d-9611-4135-bb72-339cfd331081 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:33.473036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:33.510389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:33.511031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.511309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.511582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.523233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:33.527840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:33.528674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:33.529436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 107/215] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:33.536251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:33.576114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67728091-1f52-45fc-beac-1154fc34bd2d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:33.586922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:33.615733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:33.616160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.616532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.616908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.630091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:33.635321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:33.636227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:33.637028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 109/216] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:33.642534 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:33.653096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b53290-cd14-4163-9986-813661ec1f22 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:33.689039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:33.689649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.689925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.690205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.690892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:33.691765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:33.693207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:33.701483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 108/217] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:33.703232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:33.734698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:33.754619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69d5a-175f-4128-a553-3a057834aa96 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:33.788792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:33.789328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.789603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.789871 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.790558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:33.790855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:33.792251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:33.800236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 110/218] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:33.803113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:33.808410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81984cf1-f29c-4fd2-8633-58d56b2cade5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:33.832540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:33.841922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8aec19-5ed9-4463-bad3-0908d004ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:33.842590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aec19-5ed9-4463-bad3-0908d004ce1d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.842875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aec19-5ed9-4463-bad3-0908d004ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.843153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aec19-5ed9-4463-bad3-0908d004ce1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.850305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aec19-5ed9-4463-bad3-0908d004ce1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:33.850305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aec19-5ed9-4463-bad3-0908d004ce1d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:33.850305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 109/219] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:33.850305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e02c-a9da-4df7-bd55-932def52b375 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.850305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e02c-a9da-4df7-bd55-932def52b375 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.850305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64e02c-a9da-4df7-bd55-932def52b375 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.850937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 110/220] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:33.851646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64e02c-a9da-4df7-bd55-932def52b375 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:33.903828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79eee-18ed-49cf-9be9-c11d6b40218e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:33.936339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed55af-2fdd-41ff-87e4-6de9260c7e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:33.937016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed55af-2fdd-41ff-87e4-6de9260c7e7e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.937297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed55af-2fdd-41ff-87e4-6de9260c7e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.937571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed55af-2fdd-41ff-87e4-6de9260c7e7e None admin] PATH_INFO: `/v3/users/4528fd169d59409aada5eea70a06c1b1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.938744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed55af-2fdd-41ff-87e4-6de9260c7e7e None admin] RBAC: Authorizing `identity:get_user(user_id=4528fd169d59409aada5eea70a06c1b1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:33.940244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed55af-2fdd-41ff-87e4-6de9260c7e7e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:33.942700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 111/221] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:33 2022] GET /identity/v3/users/4528fd169d59409aada5eea70a06c1b1 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:33.943648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495639f8-1780-4a82-ac55-16d49f32d454 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.943942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495639f8-1780-4a82-ac55-16d49f32d454 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.944214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495639f8-1780-4a82-ac55-16d49f32d454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:33.952950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:33.960540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64e02c-a9da-4df7-bd55-932def52b375 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:33.961977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-495639f8-1780-4a82-ac55-16d49f32d454 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:33.963192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-495639f8-1780-4a82-ac55-16d49f32d454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:33.994002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:33.994657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:33.994939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:33.995193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] PATH_INFO: `/v3/users/5104322ff2e649f695853bb43cc572ee` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.008585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] RBAC: Authorizing `identity:get_user(user_id=5104322ff2e649f695853bb43cc572ee)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:34.010026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:34.016175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 111/222] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:33 2022] GET /identity/v3/users/5104322ff2e649f695853bb43cc572ee => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:34.016175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.031473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 112/223] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:34.032309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495639f8-1780-4a82-ac55-16d49f32d454 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.061141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:34.119408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c33985-5c5e-4b0e-9ce8-aff04425b35f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:34.133097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495639f8-1780-4a82-ac55-16d49f32d454 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:34.156061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:34.156702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:34.156974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:34.157252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.157882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:34.159421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:34.173792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:34.174496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:34.174791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:34.175127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.176913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:34.178378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:34.182146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 113/224] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:34 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:34.182146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.272474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:34.278007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9ec4c-a62d-41b5-96c2-ca823067506e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:34.310517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:34.311123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:34.311454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:34.311674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.375247 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:34.376686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:34.470932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 114/225] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:34.471737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.478132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 112/226] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:45:34 2022] POST /identity/v3/users => generated 297 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:34.480086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.494079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:34.515747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:34.590136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120cf66-1b38-483b-9eb7-237cf696c90e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:34.592824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada785-0593-4d9b-87fb-2bd4e17c3cac None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:34.628226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:34.629579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:34.629865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:34.631047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/4528fd169d59409aada5eea70a06c1b1/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.632168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8c6befab82a430b8b4f72131c2e9e57, user_id=4528fd169d59409aada5eea70a06c1b1, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:34.633582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:34.633712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:34.634219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:34.634528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:34.634802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.646387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:34.651661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:34.652759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 113/227] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:34.653761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.700101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 115/228] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:34 2022] PUT /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/4528fd169d59409aada5eea70a06c1b1/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:34.701924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.730761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:34.777519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78449a57-be3e-4b31-a557-01ef0ef0c5aa None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:34.790220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:34.829816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:34.830936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:34.836420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:34.836420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.836420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:34.836420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:34.837513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 114/229] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:34 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:34.844007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:34.862098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11542b-6ec8-4555-95a6-872d8531cea9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:34.910857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:34.911692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:34.911974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:34.912242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:34.912917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:34.913218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:34.914676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:34.928304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 116/230] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:34.934501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:35.000866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54087ef0-a7ef-4e7f-987e-93b99959f9d6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:35.050414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:35.051898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:35.052606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:35.053061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:35.054295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=5104322ff2e649f695853bb43cc572ee) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:35.055240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:35.057410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:35.060608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77085948-2d54-4cf5-a0fa-5eebf39378a2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:35.069934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 115/231] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:34 2022] GET /identity/v3/role_assignments?user.id=5104322ff2e649f695853bb43cc572ee&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:35.092150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:35.126904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:35.126904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:35.126904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:35.126904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] PATH_INFO: `/v3/users/4528fd169d59409aada5eea70a06c1b1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:35.127349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] RBAC: Authorizing `identity:get_user(user_id=4528fd169d59409aada5eea70a06c1b1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:35.127349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:35.127349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 117/232] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:34 2022] GET /identity/v3/users/4528fd169d59409aada5eea70a06c1b1 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:35.130900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:35.134150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:35.267815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06b8f3-6e5b-4534-9781-e28a659a014f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:35.304866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbddb31-3f91-44d9-8399-9a2845827b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:35.305969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbddb31-3f91-44d9-8399-9a2845827b78 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:35.306503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbddb31-3f91-44d9-8399-9a2845827b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:35.306999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbddb31-3f91-44d9-8399-9a2845827b78 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:35.308348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbddb31-3f91-44d9-8399-9a2845827b78 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:35.310061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbddb31-3f91-44d9-8399-9a2845827b78 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:35.311962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 116/233] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:35 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:35.321984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:35.455785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445361-6a57-4bca-bb89-a978ef731984 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:35.491235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c50c64-68f9-4af1-83f4-20b07533ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:35.491912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50c64-68f9-4af1-83f4-20b07533ca6d None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:35.494013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50c64-68f9-4af1-83f4-20b07533ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:35.494013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c50c64-68f9-4af1-83f4-20b07533ca6d None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/4528fd169d59409aada5eea70a06c1b1/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:35.494706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50c64-68f9-4af1-83f4-20b07533ca6d None admin] RBAC: Authorizing `identity:create_grant(project_id=d8c6befab82a430b8b4f72131c2e9e57, user_id=4528fd169d59409aada5eea70a06c1b1, role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:35.496166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c50c64-68f9-4af1-83f4-20b07533ca6d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:35.620986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 118/234] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:35 2022] PUT /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/4528fd169d59409aada5eea70a06c1b1/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:36.836970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753281a3-de87-4954-8353-864ba9c6762f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:36.838012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753281a3-de87-4954-8353-864ba9c6762f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:36.838530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753281a3-de87-4954-8353-864ba9c6762f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:36.840357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 117/235] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:36.846325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d610a-3b28-44cd-8a7e-a47f87c27b4a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:36.847280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d610a-3b28-44cd-8a7e-a47f87c27b4a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:36.847977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d610a-3b28-44cd-8a7e-a47f87c27b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:36.875847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-be6d610a-3b28-44cd-8a7e-a47f87c27b4a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:36.877052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-be6d610a-3b28-44cd-8a7e-a47f87c27b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:36.934779 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:36.972967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 119/236] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:37.076674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9560-8337-45d1-b028-32cee59759ad None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.077321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9560-8337-45d1-b028-32cee59759ad None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.077808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9560-8337-45d1-b028-32cee59759ad None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.079636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 118/237] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:37.089872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e4f01-10a9-4066-b1b9-e517b5ccbf02 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.090101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e4f01-10a9-4066-b1b9-e517b5ccbf02 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.090383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e4f01-10a9-4066-b1b9-e517b5ccbf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.120157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-871e4f01-10a9-4066-b1b9-e517b5ccbf02 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:37.121350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-871e4f01-10a9-4066-b1b9-e517b5ccbf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:37.218206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 120/238] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:37.295798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a9560-8337-45d1-b028-32cee59759ad None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:37.304007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bf88f-b66c-4e29-a2f1-d7d452d3e6bb None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.304301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bf88f-b66c-4e29-a2f1-d7d452d3e6bb None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.304578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bf88f-b66c-4e29-a2f1-d7d452d3e6bb None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.306362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 121/239] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:37.311368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b1d97-095c-4bd8-87d0-45d54bc225b8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.311652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b1d97-095c-4bd8-87d0-45d54bc225b8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.311927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b1d97-095c-4bd8-87d0-45d54bc225b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.340577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-7a6b1d97-095c-4bd8-87d0-45d54bc225b8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:37.341695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7a6b1d97-095c-4bd8-87d0-45d54bc225b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:37.422722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:37.455777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 122/240] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:37.465090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a9560-8337-45d1-b028-32cee59759ad None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:37.500088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:37.500717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.500991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.501255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.512287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:37.517362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:37.518215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:37.519108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 119/241] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:37.525608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b1d97-095c-4bd8-87d0-45d54bc225b8 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:37.577621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:37.580893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:37.694897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d30df-585f-4685-a610-ae9ebb087d0b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:37.711623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b1d97-095c-4bd8-87d0-45d54bc225b8 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:37.723581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:37.735314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:37.735951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.738759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.738759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.761840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:37.766715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:37.767814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 120/242] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:37.769573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:37.772408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:37.773358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.773713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.774063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.775020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:37.775433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:37.777837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:37.788237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 123/243] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:37.790720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:37.890082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90434b-475f-42f6-8069-5ea0c4f58222 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:37.914784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:37.918460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f31bc-0f36-4e23-a10e-7b6b24225c70 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:37.928225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:37.933218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.933218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.933218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.933218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:37.933218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:37.933681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 121/244] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:37.934472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:37.957795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d0181-f198-48e7-8c29-189eea6f0430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:37.958360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0181-f198-48e7-8c29-189eea6f0430 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.958639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0181-f198-48e7-8c29-189eea6f0430 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.958908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d0181-f198-48e7-8c29-189eea6f0430 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.959566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d0181-f198-48e7-8c29-189eea6f0430 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:37.959861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d0181-f198-48e7-8c29-189eea6f0430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:37.961238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d0181-f198-48e7-8c29-189eea6f0430 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:37.970301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 124/245] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:37.971509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ead7-03e1-495a-ad8c-2c126daa8780 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:37.971798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ead7-03e1-495a-ad8c-2c126daa8780 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:37.972634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10ead7-03e1-495a-ad8c-2c126daa8780 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:37.974551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 125/246] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:37.975246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10ead7-03e1-495a-ad8c-2c126daa8780 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:38.007399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:38.051383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881c192-fa75-41ef-a927-e18b3bae865d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:38.087572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10ead7-03e1-495a-ad8c-2c126daa8780 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:38.088176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:38.088807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.089093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.089369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] PATH_INFO: `/v3/users/acdebe43b03c46beb1ea177899802bd5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.105460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] RBAC: Authorizing `identity:get_user(user_id=acdebe43b03c46beb1ea177899802bd5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:38.106923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:38.108876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 122/247] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:37 2022] GET /identity/v3/users/acdebe43b03c46beb1ea177899802bd5 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:38.109654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:38.127731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd55237-ad70-40ee-ae6e-d867e10822f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:38.128385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55237-ad70-40ee-ae6e-d867e10822f9 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.128757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55237-ad70-40ee-ae6e-d867e10822f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.129053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55237-ad70-40ee-ae6e-d867e10822f9 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.129740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd55237-ad70-40ee-ae6e-d867e10822f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:38.131235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd55237-ad70-40ee-ae6e-d867e10822f9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:38.201752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 126/248] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:45:37 2022] POST /identity/v3/users => generated 293 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:38.201752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e36d9-7610-4cfb-b240-dd0ca59128c4 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.201752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e36d9-7610-4cfb-b240-dd0ca59128c4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.201752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e36d9-7610-4cfb-b240-dd0ca59128c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.222646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bea225-df30-41e3-85ba-e6f868f097bc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:38.229408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-bc8e36d9-7610-4cfb-b240-dd0ca59128c4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:38.230835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bc8e36d9-7610-4cfb-b240-dd0ca59128c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:38.235571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:38.245542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:38.259254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:38.259909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.260187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.260460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] PATH_INFO: `/v3/users/4528fd169d59409aada5eea70a06c1b1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.261700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] RBAC: Authorizing `identity:get_user(user_id=4528fd169d59409aada5eea70a06c1b1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:38.263257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:38.271361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 123/249] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:38 2022] GET /identity/v3/users/4528fd169d59409aada5eea70a06c1b1 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:38.271361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:38.326267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 127/250] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:38.326267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75fb2d-ac3b-4bad-8f3f-9180ca77906f None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.326267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75fb2d-ac3b-4bad-8f3f-9180ca77906f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.326267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75fb2d-ac3b-4bad-8f3f-9180ca77906f None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.326267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 128/251] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:38.326710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75fb2d-ac3b-4bad-8f3f-9180ca77906f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:38.398056 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28193a-0f5d-4b85-bd80-e921b7db0de6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:38.441679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01882f5a-637f-4b18-889c-989c1b668475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:38.442326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01882f5a-637f-4b18-889c-989c1b668475 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.442625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01882f5a-637f-4b18-889c-989c1b668475 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.442882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01882f5a-637f-4b18-889c-989c1b668475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.463986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75fb2d-ac3b-4bad-8f3f-9180ca77906f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:38.509153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:38.511629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:38.512535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.512535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.512872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.528204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01882f5a-637f-4b18-889c-989c1b668475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:38.530006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01882f5a-637f-4b18-889c-989c1b668475 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:38.535995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:38.616551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:38.618091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:38.669863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 124/252] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:38.671728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29518939-3fb5-42d4-b3a6-98463e2c65a5 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.672010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29518939-3fb5-42d4-b3a6-98463e2c65a5 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.672576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29518939-3fb5-42d4-b3a6-98463e2c65a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.694488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-29518939-3fb5-42d4-b3a6-98463e2c65a5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:38.695733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-29518939-3fb5-42d4-b3a6-98463e2c65a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:38.742562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 129/253] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:38.747034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:38.776535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 125/254] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:38.777351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29518939-3fb5-42d4-b3a6-98463e2c65a5 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:38.805850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:38.825877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:38.869781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf5bf0-0e16-4737-8e94-c646cb56ec50 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:38.914782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:38.915428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:38.915959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:38.915959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:38.943262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:38.949143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:38.950005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 130/255] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:38.951874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:38.999118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29518939-3fb5-42d4-b3a6-98463e2c65a5 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.056887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.057910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.058435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.058910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.060975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:39.062656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:39.074849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 126/256] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:38 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:39.076045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.139883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60342786-4890-4a19-8078-ca03f44cb9d6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.146995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:39.176002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.176652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.176929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.177198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.178576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:39.181838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:39.183280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:39.188674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 131/257] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:38 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:39.189516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.207421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53027a0b-dce6-4a14-8794-05dee8b0787f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.246692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.247852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.248332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.248818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.249736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] RBAC: Adding query filter params (name=member) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:39.250442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:39.252153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:39.261994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 127/258] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:39.263111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.303091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f129-033c-4e54-b5bf-285c9f223289 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.344697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.345232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.345511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.345785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.346539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] RBAC: Adding query filter params (role.id=d2c9d0ae0229447bb2c52d38aa7e005c, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=acdebe43b03c46beb1ea177899802bd5) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:39.347367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:39.348968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:39.355098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 132/259] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/role_assignments?user.id=acdebe43b03c46beb1ea177899802bd5&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=d2c9d0ae0229447bb2c52d38aa7e005c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:39.356214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.397097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494242-dc35-42c8-8a1a-1a60b95d8863 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.403599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:39.459869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.460510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.460853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.461121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.472313 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3b980-e8b6-411a-8b7e-d05155965fde None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.473700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:39.479367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:39.480235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:39.481132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 128/260] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:39.488583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.493084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:39.511800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.512691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.513118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.513569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.514516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=4528fd169d59409aada5eea70a06c1b1) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:39.515016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:39.516864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:39.528709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 133/261] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/role_assignments?user.id=4528fd169d59409aada5eea70a06c1b1&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:39.530290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.684067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1ae6c-72c9-438f-9e7b-6030f3e90d22 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.728940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f635b-deaa-4d90-a228-f5f8c4c84399 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:39.730766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:39.755372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.756026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.756312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.756585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] PATH_INFO: `/v3/users/5104322ff2e649f695853bb43cc572ee` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.758094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] RBAC: Authorizing `identity:get_user(user_id=5104322ff2e649f695853bb43cc572ee)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:39.759580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:39.761177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 129/262] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/users/5104322ff2e649f695853bb43cc572ee => generated 297 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:39.770843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.799918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:39.800498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:39.800785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:39.801059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:39.801729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:39.802028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:39.803435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:39.820595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 134/263] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:39.826278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:39.906573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:39.972970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9da8cf-3013-446b-9a8b-3b8ee2a00664 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:40.026085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52ab94-5f5c-43c6-a0e8-274d09e56d31 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:40.034138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:40.035441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.035716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.036687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.037119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:40.038579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:40.040524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 130/264] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:40.046052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:40.093122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c821c-632c-470e-8056-10fc30482320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:40.093782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c821c-632c-470e-8056-10fc30482320 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.094063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c821c-632c-470e-8056-10fc30482320 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.094360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c821c-632c-470e-8056-10fc30482320 None admin] PATH_INFO: `/v3/users/4528fd169d59409aada5eea70a06c1b1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.095647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c821c-632c-470e-8056-10fc30482320 None admin] RBAC: Authorizing `identity:get_user(user_id=4528fd169d59409aada5eea70a06c1b1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:40.097996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c821c-632c-470e-8056-10fc30482320 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:40.100787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 135/265] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:39 2022] GET /identity/v3/users/4528fd169d59409aada5eea70a06c1b1 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:40.106331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c821c-632c-470e-8056-10fc30482320 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:40.166000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:40.245792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e718f-6518-49ca-b55f-7b01d575f32f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:40.301122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c821c-632c-470e-8056-10fc30482320 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:40.318117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:40.318812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.319095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.319373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/5104322ff2e649f695853bb43cc572ee/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.323140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8c6befab82a430b8b4f72131c2e9e57, user_id=5104322ff2e649f695853bb43cc572ee, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:40.324595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:40.352834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:40.353462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.353743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.354014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.392327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:40.397312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 131/266] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:40 2022] PUT /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/5104322ff2e649f695853bb43cc572ee/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:40.433277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:40.434939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:40.467978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:40.543895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 136/267] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:40.549471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:40.681233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63382428-e922-49c6-809c-9fdf086dc592 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:40.719061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8d10-6215-42bc-89a2-b6ff1cdddab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:40.720045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8d10-6215-42bc-89a2-b6ff1cdddab6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.720525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8d10-6215-42bc-89a2-b6ff1cdddab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.720993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8d10-6215-42bc-89a2-b6ff1cdddab6 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.722912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b8d10-6215-42bc-89a2-b6ff1cdddab6 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:40.724755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b8d10-6215-42bc-89a2-b6ff1cdddab6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:40.726687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 132/268] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:40 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:40.735352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:40.778863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d07c4c-180a-4f62-81e2-9e0677323e7b None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.779503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d07c4c-180a-4f62-81e2-9e0677323e7b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.779979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d07c4c-180a-4f62-81e2-9e0677323e7b None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.781527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 133/269] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:40.796402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c168911-08d2-4429-89af-a937ef5efbb7 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.796989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c168911-08d2-4429-89af-a937ef5efbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.797508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c168911-08d2-4429-89af-a937ef5efbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.822401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6c168911-08d2-4429-89af-a937ef5efbb7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:40.823661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6c168911-08d2-4429-89af-a937ef5efbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:40.863815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbab79-274e-4795-8960-4e9b06e6dfb8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:40.897684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:40.904028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:40.905111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.905590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:40.906075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.906957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.907911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] RBAC: Adding query filter params (role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=4528fd169d59409aada5eea70a06c1b1) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:40.908397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:40.910175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:40.911232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 134/270] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:40.912573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ebcaf-e6d9-4c03-8c41-c417cdb7c45e None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.912948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ebcaf-e6d9-4c03-8c41-c417cdb7c45e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.913270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ebcaf-e6d9-4c03-8c41-c417cdb7c45e None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.916008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 135/271] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:40.930417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 137/272] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:40 2022] GET /identity/v3/role_assignments?user.id=4528fd169d59409aada5eea70a06c1b1&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:40.930550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859551-c4d3-4673-a778-7a4a59d3a36e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:40.930550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859551-c4d3-4673-a778-7a4a59d3a36e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:40.930550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a859551-c4d3-4673-a778-7a4a59d3a36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:40.951224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8a859551-c4d3-4673-a778-7a4a59d3a36e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:40.953105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8a859551-c4d3-4673-a778-7a4a59d3a36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:41.033899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 136/273] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:41.305110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:41.334135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859551-c4d3-4673-a778-7a4a59d3a36e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:41.427596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b40b4-d984-438c-941e-d031c700f140 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:41.442131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a859551-c4d3-4673-a778-7a4a59d3a36e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:41.468876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:41.469876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:41.470388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:41.470857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:41.471367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:41.480412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:41.481334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:41.481334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:41.481612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:41.482725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:41.484163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:41.485398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:41.485738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 137/274] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:41.491196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:41.495208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:41.496216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:41.497437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 138/275] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:41.501772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:41.591484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:41.627482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16122ba8-e805-44a0-b720-fbcd03f36678 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:41.658878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2866372-3cd6-489e-b9f3-6e6a22698e2f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:41.675858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f241a-d66e-41f8-920d-d19d13e452e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:41.677374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f241a-d66e-41f8-920d-d19d13e452e7 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:41.678071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f241a-d66e-41f8-920d-d19d13e452e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:41.678591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f241a-d66e-41f8-920d-d19d13e452e7 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:41.679451 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f241a-d66e-41f8-920d-d19d13e452e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:41.681171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f241a-d66e-41f8-920d-d19d13e452e7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:41.695395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:41.696057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:41.696339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:41.696660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:41.697361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:41.697663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:41.699168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:41.708630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 139/276] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:41.713929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:41.791881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 138/277] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:45:41 2022] POST /identity/v3/users => generated 293 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:45:41.793613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a22f8-478b-4e0e-a961-99697e9b1d0b None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:41.794166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a22f8-478b-4e0e-a961-99697e9b1d0b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:41.794673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a22f8-478b-4e0e-a961-99697e9b1d0b None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:41.796154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 139/278] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:41.802698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d007c-095e-4666-b648-cf92502ce587 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:41.802986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d007c-095e-4666-b648-cf92502ce587 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:41.805181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d007c-095e-4666-b648-cf92502ce587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:41.826386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4b8d007c-095e-4666-b648-cf92502ce587 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:41.827934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4b8d007c-095e-4666-b648-cf92502ce587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:41.829543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c894f-6290-4a55-95b8-3e330e1ecac2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:41.850190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:41.882995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:41.883659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:41.883940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:41.884232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] PATH_INFO: `/v3/users/e1898741e88f45f9b2116250262bd4d1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:41.891961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:41.899438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] RBAC: Authorizing `identity:get_user(user_id=e1898741e88f45f9b2116250262bd4d1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:41.900928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:41.903079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 140/279] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:41 2022] GET /identity/v3/users/e1898741e88f45f9b2116250262bd4d1 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:41.908855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:41.916276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 140/280] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:42.025017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254412f-18aa-4d03-b47f-03c9b7fc668a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:42.061148 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:42.061813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:42.062044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:42.062381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:42.063108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, user.id=e1898741e88f45f9b2116250262bd4d1, scope.system=all) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:42.063409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:42.064833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:42.071139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 141/281] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 22 10:45:41 2022] GET /identity/v3/role_assignments?user.id=e1898741e88f45f9b2116250262bd4d1&scope.system=all&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:42.199125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d007c-095e-4666-b648-cf92502ce587 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:42.314089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d007c-095e-4666-b648-cf92502ce587 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:42.356609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:42.357243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:42.357515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:42.357784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:42.379933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:42.384804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:42.385664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:42.386854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 141/282] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:42.393666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:42.460453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:42.534954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305c099-8b9e-4fe5-bb57-8690163c7f88 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:42.585919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbfb4-c8af-480d-8337-b8204a253f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:42.586589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbfb4-c8af-480d-8337-b8204a253f6b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:42.586869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbfb4-c8af-480d-8337-b8204a253f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:42.587141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbfb4-c8af-480d-8337-b8204a253f6b None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:42.587808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbfb4-c8af-480d-8337-b8204a253f6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:42.588107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbfb4-c8af-480d-8337-b8204a253f6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:42.589593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbfb4-c8af-480d-8337-b8204a253f6b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:42.601652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 142/283] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:42.607458 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:42.630668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bcfd7-4835-46fd-94f9-f8c3a1d1743a None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:42.630994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bcfd7-4835-46fd-94f9-f8c3a1d1743a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:42.631218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bcfd7-4835-46fd-94f9-f8c3a1d1743a None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:42.632812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 143/284] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:42.637696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556d3f-a746-46b7-8f4d-448ce48f6d2e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:42.637983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556d3f-a746-46b7-8f4d-448ce48f6d2e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:42.638304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556d3f-a746-46b7-8f4d-448ce48f6d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:42.658133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c2556d3f-a746-46b7-8f4d-448ce48f6d2e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:42.660396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c2556d3f-a746-46b7-8f4d-448ce48f6d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:42.712263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:42.773721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 144/285] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:42.779977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88f738-86d5-406d-8b19-715fe4fd6003 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:42.837889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:42.838558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:42.838838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:42.839112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] PATH_INFO: `/v3/users/acdebe43b03c46beb1ea177899802bd5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:42.840253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] RBAC: Authorizing `identity:get_user(user_id=acdebe43b03c46beb1ea177899802bd5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:42.841700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:42.843689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 142/286] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:42 2022] GET /identity/v3/users/acdebe43b03c46beb1ea177899802bd5 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:42.849680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556d3f-a746-46b7-8f4d-448ce48f6d2e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:42.894253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:42.988769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556d3f-a746-46b7-8f4d-448ce48f6d2e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:43.030040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:43.030705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:43.030983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:43.031255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:43.035346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:43.036802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:43.038609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 145/287] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:42 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:43.044255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:43.183007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:43.186992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f60bb-0ead-4174-8942-49f8ceed9920 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:43.205599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:43.239796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:43.240797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:43.241270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:43.241775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/acdebe43b03c46beb1ea177899802bd5/roles/d2c9d0ae0229447bb2c52d38aa7e005c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:43.247006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8c6befab82a430b8b4f72131c2e9e57, user_id=acdebe43b03c46beb1ea177899802bd5, role_id=d2c9d0ae0229447bb2c52d38aa7e005c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:43.248808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:43.290827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:43.294056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19885fff-d35f-4559-88cf-bd352e25b299 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:43.329089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:43.329785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:43.330068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:43.330364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:43.344512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 143/288] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:45:43 2022] PUT /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/users/acdebe43b03c46beb1ea177899802bd5/roles/d2c9d0ae0229447bb2c52d38aa7e005c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:43.346126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:43.351849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:43.352771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:43.353535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 146/289] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:43.359428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:43.498971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44425b24-f4e6-475d-80a5-6c5fb3aae887 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:43.545533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:43.546151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:43.546467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:43.546747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:43.547412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] RBAC: Adding query filter params (name=member) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:43.547714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:43.549086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:43.558110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 144/290] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:43.563712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:43.667561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865f4c9-1046-4b87-93b1-45da36b0940c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:43.702858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:43.703499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:43.703782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:43.704055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] PATH_INFO: `/v3/users/5104322ff2e649f695853bb43cc572ee` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:43.705183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] RBAC: Authorizing `identity:get_user(user_id=5104322ff2e649f695853bb43cc572ee)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:43.706833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:43.708406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 147/291] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:43 2022] GET /identity/v3/users/5104322ff2e649f695853bb43cc572ee => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:43.714161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:43.752296 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:43.834694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16df89-e74f-419c-9921-236b4a831f8b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:43.891164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:43.891782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:43.892059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:43.892331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:43.980220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:43.981688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:44.025567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:44.115442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 145/292] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:44.121048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:44.151265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:44.225419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444d793-6194-4398-9877-e1865f2099b7 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:44.262084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee3659-3ae5-4c6f-bdbc-1554124b29a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:44.263147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3659-3ae5-4c6f-bdbc-1554124b29a0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.263670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3659-3ae5-4c6f-bdbc-1554124b29a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.264138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee3659-3ae5-4c6f-bdbc-1554124b29a0 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.265450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee3659-3ae5-4c6f-bdbc-1554124b29a0 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:44.267102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee3659-3ae5-4c6f-bdbc-1554124b29a0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:44.269795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 148/293] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:44 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:44.275913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:44.400291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e6667-593c-419f-9f18-3b9f7ab8bcd5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:44.443644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:44.449494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf8dfe-a776-450c-92db-d985f368f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:44.450380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8dfe-a776-450c-92db-d985f368f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.450983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8dfe-a776-450c-92db-d985f368f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.451479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf8dfe-a776-450c-92db-d985f368f7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.452431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8dfe-a776-450c-92db-d985f368f7b4 None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=5104322ff2e649f695853bb43cc572ee) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:44.452896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8dfe-a776-450c-92db-d985f368f7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:44.454511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf8dfe-a776-450c-92db-d985f368f7b4 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:44.464925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 146/294] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:44 2022] GET /identity/v3/role_assignments?user.id=5104322ff2e649f695853bb43cc572ee&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:44.520358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee02e4-333a-496d-a8b8-b689cddb1737 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.520926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee02e4-333a-496d-a8b8-b689cddb1737 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.521422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee02e4-333a-496d-a8b8-b689cddb1737 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.523023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 149/295] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:44.529131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598c210-1db3-4c16-946d-8babf13951b2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.529676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598c210-1db3-4c16-946d-8babf13951b2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.530111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598c210-1db3-4c16-946d-8babf13951b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.558820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b598c210-1db3-4c16-946d-8babf13951b2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:44.560377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b598c210-1db3-4c16-946d-8babf13951b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:44.567918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7a196-669e-4267-8ab4-c6cdb5feba0e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.568762 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7a196-669e-4267-8ab4-c6cdb5feba0e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.569251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7a196-669e-4267-8ab4-c6cdb5feba0e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.570914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 150/296] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:44.572356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c84d7-b106-43cc-80f7-7eec7db859cd None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.572966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c84d7-b106-43cc-80f7-7eec7db859cd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.573450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c84d7-b106-43cc-80f7-7eec7db859cd None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.575080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 151/297] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:44.578221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1d18b-a378-4ad1-b926-d3d70d01fe28 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.578780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1d18b-a378-4ad1-b926-d3d70d01fe28 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.579246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1d18b-a378-4ad1-b926-d3d70d01fe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.600944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2db1d18b-a378-4ad1-b926-d3d70d01fe28 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:44.602931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2db1d18b-a378-4ad1-b926-d3d70d01fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:44.641565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 147/298] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:44.642839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7b66b-dd16-47fe-b7cc-6858a55fbaa9 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:44.643121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7b66b-dd16-47fe-b7cc-6858a55fbaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:44.643421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7b66b-dd16-47fe-b7cc-6858a55fbaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:44.676533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-85c7b66b-dd16-47fe-b7cc-6858a55fbaa9 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:44.677922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-85c7b66b-dd16-47fe-b7cc-6858a55fbaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:44.691637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 152/299] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:44.764686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 148/300] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:44.939152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1d18b-a378-4ad1-b926-d3d70d01fe28 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:44.968175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:44.994591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7b66b-dd16-47fe-b7cc-6858a55fbaa9 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.053896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1d18b-a378-4ad1-b926-d3d70d01fe28 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.072789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:45.096989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.097708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.097988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.098278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.108667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:45.113424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:45.114300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:45.115437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 153/301] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:45.116072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.124711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c7b66b-dd16-47fe-b7cc-6858a55fbaa9 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.161889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.162556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.162839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.163116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.164227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.165669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.171906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 149/302] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:45.172663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.209836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:45.226572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c592e5-95d0-4a09-b0a0-4d3cd1e1d00e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.262875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.263567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.263853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.264121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.273306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858635b-dbfc-46a3-8d28-b62f1b554cba None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.275341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:45.280064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:45.280912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:45.281689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:45.287656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 154/303] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:45.289192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.301517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:45.312015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.312627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.312906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.313180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.313848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] RBAC: Adding query filter params (name=member) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:45.320403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.320403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.324577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 150/304] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:45.329649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.401469 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe14d1a-170d-44cb-8900-dba56c042d39 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.436324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.436944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.437226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.437498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] PATH_INFO: `/v3/groups` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.437774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d617-f663-4bcf-9cfd-c19a93d02633 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.438217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.439576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.472980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.473591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.473877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.474147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.474841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:45.475140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.476534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.484480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 151/305] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:45.486402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.518256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 155/306] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 22 10:45:45 2022] POST /identity/v3/groups => generated 229 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:45.518256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.535904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:45.538250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:45.608936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59ebc3-db0d-4e04-b37a-8955f1cf83a9 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.651863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.652611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.652888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.653163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] PATH_INFO: `/v3/users/e1898741e88f45f9b2116250262bd4d1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.654422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] RBAC: Authorizing `identity:get_user(user_id=e1898741e88f45f9b2116250262bd4d1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.656087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.659742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 152/307] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:45 2022] GET /identity/v3/users/e1898741e88f45f9b2116250262bd4d1 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:45.665074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.670899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817efc8-93c8-40f8-9057-f0675f81ab30 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.718572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd767a-f25b-42f8-9a6e-3c8dbfc7a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.719224 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd767a-f25b-42f8-9a6e-3c8dbfc7a3df None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.719620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd767a-f25b-42f8-9a6e-3c8dbfc7a3df None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.720012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd767a-f25b-42f8-9a6e-3c8dbfc7a3df None admin] PATH_INFO: `/v3/users/0f70eec8436d4e2fa55791f2951e86fd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.735225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd767a-f25b-42f8-9a6e-3c8dbfc7a3df None admin] RBAC: Authorizing `identity:get_user(user_id=0f70eec8436d4e2fa55791f2951e86fd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.736672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd767a-f25b-42f8-9a6e-3c8dbfc7a3df None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.742251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 156/308] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:45 2022] GET /identity/v3/users/0f70eec8436d4e2fa55791f2951e86fd => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:45.742251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea4ea-4fcc-49dc-b5f2-6b775956f9e2 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.742251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea4ea-4fcc-49dc-b5f2-6b775956f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.742251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bea4ea-4fcc-49dc-b5f2-6b775956f9e2 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.742251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 157/309] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:45.747116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bea4ea-4fcc-49dc-b5f2-6b775956f9e2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:45.778995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46ced5-3550-4ba1-af93-9fbd55b9ca76 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.795798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:45.814643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.815844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.816121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.816563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] PATH_INFO: `/v3/system/users/e1898741e88f45f9b2116250262bd4d1/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.817433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:45.817807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e1898741e88f45f9b2116250262bd4d1, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.819203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.855804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bea4ea-4fcc-49dc-b5f2-6b775956f9e2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:45.890458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a23943-0730-4bcb-92aa-cc3a2595234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:45.891075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23943-0730-4bcb-92aa-cc3a2595234a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.891343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23943-0730-4bcb-92aa-cc3a2595234a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.891613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a23943-0730-4bcb-92aa-cc3a2595234a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.892329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23943-0730-4bcb-92aa-cc3a2595234a None admin] RBAC: Adding query filter params (role.id=d2c9d0ae0229447bb2c52d38aa7e005c, user.id=0f70eec8436d4e2fa55791f2951e86fd, scope.system=all) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:45.892633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23943-0730-4bcb-92aa-cc3a2595234a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:45.894697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a23943-0730-4bcb-92aa-cc3a2595234a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:45.899526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 158/310] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 22 10:45:45 2022] GET /identity/v3/role_assignments?user.id=0f70eec8436d4e2fa55791f2951e86fd&scope.system=all&role.id=d2c9d0ae0229447bb2c52d38aa7e005c => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:45.900660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8300e95-5876-41f1-8ec7-9e8db3ecbaf0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:45.900941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8300e95-5876-41f1-8ec7-9e8db3ecbaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:45.901212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8300e95-5876-41f1-8ec7-9e8db3ecbaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:45.905352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 153/311] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 22 10:45:45 2022] PUT /identity/v3/system/users/e1898741e88f45f9b2116250262bd4d1/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 10:45:45.926923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b8300e95-5876-41f1-8ec7-9e8db3ecbaf0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:45.928705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b8300e95-5876-41f1-8ec7-9e8db3ecbaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:46.015436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 159/312] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:46.287991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:46.391224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b1f8b-09f9-4f15-9a55-d61dddb02b0f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:46.426698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:46.427653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:46.428113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:46.428556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:46.439103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:46.444929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:46.445774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:46.446560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:46.448786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 154/313] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:46.454424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8300e95-5876-41f1-8ec7-9e8db3ecbaf0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:46.514603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:46.560727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8300e95-5876-41f1-8ec7-9e8db3ecbaf0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:46.595657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:46.597066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:46.597481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:46.597886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:46.598704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:46.599154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:46.600678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:46.609747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 160/314] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:46.615695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:46.657971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:46.719848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df56ca2-a2b9-44d6-890c-b107b8d39cc2 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:46.753735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:46.754392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:46.754673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:46.754955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] PATH_INFO: `/v3/users/acdebe43b03c46beb1ea177899802bd5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:46.756197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] RBAC: Authorizing `identity:get_user(user_id=acdebe43b03c46beb1ea177899802bd5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:46.757967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:46.760279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 155/315] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:46 2022] GET /identity/v3/users/acdebe43b03c46beb1ea177899802bd5 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:46.767335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:46.889191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53e941-caec-4585-9a37-2483e8fdbd7d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:46.911010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:46.927554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:46.928172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:46.928455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:46.931123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.000002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:47.001577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:47.108692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 161/316] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:47.114046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:47.214670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4479b-0618-4500-b256-d4e560ef3892 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:47.224423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:47.248874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaa7c1-b362-492b-9ad7-4e5a0eff93d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:47.249503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaa7c1-b362-492b-9ad7-4e5a0eff93d4 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.249783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaa7c1-b362-492b-9ad7-4e5a0eff93d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.250054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaa7c1-b362-492b-9ad7-4e5a0eff93d4 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.251208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaa7c1-b362-492b-9ad7-4e5a0eff93d4 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:47.252707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaa7c1-b362-492b-9ad7-4e5a0eff93d4 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:47.254464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 156/317] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:47 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:47.260479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:47.282721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:47.367377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0306a-a48a-4bd9-b568-c1a944bfcf39 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:47.403276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad6950-397b-4320-a63c-8f6a552dfbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:47.404252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad6950-397b-4320-a63c-8f6a552dfbcc None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.404731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad6950-397b-4320-a63c-8f6a552dfbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.405196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad6950-397b-4320-a63c-8f6a552dfbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.406111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad6950-397b-4320-a63c-8f6a552dfbcc None admin] RBAC: Adding query filter params (role.id=d2c9d0ae0229447bb2c52d38aa7e005c, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57, user.id=acdebe43b03c46beb1ea177899802bd5) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:47.406673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad6950-397b-4320-a63c-8f6a552dfbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:47.408305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad6950-397b-4320-a63c-8f6a552dfbcc None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:47.415530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 162/318] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:45:47 2022] GET /identity/v3/role_assignments?user.id=acdebe43b03c46beb1ea177899802bd5&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=d2c9d0ae0229447bb2c52d38aa7e005c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:47.584723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f646-341c-4368-8b10-e2833159f701 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.585212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f646-341c-4368-8b10-e2833159f701 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.585463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26f646-341c-4368-8b10-e2833159f701 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.587034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 157/319] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:47.592463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491d6c5-7524-4103-b1bc-a1967214f7e3 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.592748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491d6c5-7524-4103-b1bc-a1967214f7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.593198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491d6c5-7524-4103-b1bc-a1967214f7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.614832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e491d6c5-7524-4103-b1bc-a1967214f7e3 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:47.616047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e491d6c5-7524-4103-b1bc-a1967214f7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:47.688763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 163/320] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:47.849593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5d5a9-37bc-4e87-8767-c294218ed68e None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.903251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5d5a9-37bc-4e87-8767-c294218ed68e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.903251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5d5a9-37bc-4e87-8767-c294218ed68e None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.903251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 158/321] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:47.904060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97128c-2d0d-40cf-981a-7b0a6e89b59b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.904060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97128c-2d0d-40cf-981a-7b0a6e89b59b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.904060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97128c-2d0d-40cf-981a-7b0a6e89b59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.904060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-dd97128c-2d0d-40cf-981a-7b0a6e89b59b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:47.904060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dd97128c-2d0d-40cf-981a-7b0a6e89b59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:47.904060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:47.919611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5882-bed6-45ae-a289-8797f75cf0e0 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.919987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5882-bed6-45ae-a289-8797f75cf0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.920330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5882-bed6-45ae-a289-8797f75cf0e0 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.921854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 159/322] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:45:47.928088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455354e2-6438-4d85-900b-edf82d6d5b96 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:47.928496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455354e2-6438-4d85-900b-edf82d6d5b96 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:47.928655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455354e2-6438-4d85-900b-edf82d6d5b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:47.947536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-455354e2-6438-4d85-900b-edf82d6d5b96 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:47.948663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-455354e2-6438-4d85-900b-edf82d6d5b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:47.953297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 164/323] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:47.959917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97128c-2d0d-40cf-981a-7b0a6e89b59b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.023194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 160/324] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:48.062755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97128c-2d0d-40cf-981a-7b0a6e89b59b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.096387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.097015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.097287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.097550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.098652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:48.100073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:48.101763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 165/325] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:45:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:48.107232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455354e2-6438-4d85-900b-edf82d6d5b96 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.204481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455354e2-6438-4d85-900b-edf82d6d5b96 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.212228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:48.229806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.238763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.239369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.239647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.239918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] PATH_INFO: `/v3/groups` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.240593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:48.242446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:48.247158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:48.332814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6c07e-552a-45db-af86-2148407a2ec2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.333277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 161/326] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 22 10:45:48 2022] POST /identity/v3/groups => generated 226 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:45:48.334096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.379309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.379976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.380254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.380522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.392594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:48.398292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:48.399453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:48.400280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 166/327] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:48.405360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.468635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124d6d2-5f7f-4385-be17-8054cc7aac22 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.486109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:48.513288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.513928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.514909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.514909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.523007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefef9f0-ead8-48ed-9b2c-60db0ae38e87 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.527654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:48.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:48.534485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:48.535374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:48.538896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 162/328] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:48.539916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.564323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.565877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.565877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.565877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.566472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:48.566776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:48.568153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:48.578298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:48.581800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 167/329] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:48.587352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.672010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23ab02-acca-4f5d-92f0-56167aed8098 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.712603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.713217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.713496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.713768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.714486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] RBAC: Adding query filter params (name=member) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:48.714787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:48.716167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:48.727786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 163/330] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:48.729734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c6cea-a493-4d93-9171-20c8d980f985 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.736509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.774408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.775062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.775338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.776324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] PATH_INFO: `/v3/users/0f70eec8436d4e2fa55791f2951e86fd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.779065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] RBAC: Authorizing `identity:get_user(user_id=0f70eec8436d4e2fa55791f2951e86fd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:48.781886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:48.784470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 168/331] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:48 2022] GET /identity/v3/users/0f70eec8436d4e2fa55791f2951e86fd => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:48.789580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.793508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:48.870621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:48.881137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4c364-1ec2-4f78-8060-02992428e44f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.920798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.921995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.922926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.923497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] PATH_INFO: `/v3/users/e1898741e88f45f9b2116250262bd4d1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.925433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] RBAC: Authorizing `identity:get_user(user_id=e1898741e88f45f9b2116250262bd4d1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:48.927008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70b3a5-c4a3-41c1-8054-d0382487e4e2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:48.927782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:48.930405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 164/332] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:48 2022] GET /identity/v3/users/e1898741e88f45f9b2116250262bd4d1 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:48.939354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:48.964431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f4ed8-5bdd-4eec-9969-7e08d4d6fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:48.965546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f4ed8-5bdd-4eec-9969-7e08d4d6fbca None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:48.965546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f4ed8-5bdd-4eec-9969-7e08d4d6fbca None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:48.965741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f4ed8-5bdd-4eec-9969-7e08d4d6fbca None admin] PATH_INFO: `/v3/system/users/0f70eec8436d4e2fa55791f2951e86fd/roles/d2c9d0ae0229447bb2c52d38aa7e005c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:48.967195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f4ed8-5bdd-4eec-9969-7e08d4d6fbca None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0f70eec8436d4e2fa55791f2951e86fd, role_id=d2c9d0ae0229447bb2c52d38aa7e005c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:48.968597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f4ed8-5bdd-4eec-9969-7e08d4d6fbca None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:49.031723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 169/333] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 22 10:45:48 2022] PUT /identity/v3/system/users/0f70eec8436d4e2fa55791f2951e86fd/roles/d2c9d0ae0229447bb2c52d38aa7e005c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:49.059346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8531d5-7a9e-4ff4-bc1a-b9c213fe3f8d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:49.105859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961f93-53c8-4519-a94c-784225f5e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:49.106864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961f93-53c8-4519-a94c-784225f5e046 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:49.107373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961f93-53c8-4519-a94c-784225f5e046 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:49.107909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961f93-53c8-4519-a94c-784225f5e046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:49.108924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961f93-53c8-4519-a94c-784225f5e046 None admin] RBAC: Adding query filter params (role.id=f4ba3d348adb4b6497d8a69e88e3822b, user.id=e1898741e88f45f9b2116250262bd4d1, scope.system=all) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:49.109497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961f93-53c8-4519-a94c-784225f5e046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:49.111311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961f93-53c8-4519-a94c-784225f5e046 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:49.126559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 165/334] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 22 10:45:48 2022] GET /identity/v3/role_assignments?user.id=e1898741e88f45f9b2116250262bd4d1&scope.system=all&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:50.964424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700e5f8-33af-47c6-8788-55c6451139da None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:50.964946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700e5f8-33af-47c6-8788-55c6451139da None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:50.965021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700e5f8-33af-47c6-8788-55c6451139da None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:50.966455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 170/335] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:50.975849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b54685-6b39-41ca-9484-b27c6bea2b03 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:50.976347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b54685-6b39-41ca-9484-b27c6bea2b03 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:50.976424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b54685-6b39-41ca-9484-b27c6bea2b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:50.999201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-32b54685-6b39-41ca-9484-b27c6bea2b03 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:51.000412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-32b54685-6b39-41ca-9484-b27c6bea2b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:51.010327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:51.075754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 166/336] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:51.199017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e91d8-7e53-4444-9a34-92b120c5c78c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:51.199305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e91d8-7e53-4444-9a34-92b120c5c78c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:51.199491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e91d8-7e53-4444-9a34-92b120c5c78c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:51.201106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 171/337] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:51.206616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860b1e9-9ae1-485e-8087-fd30013e87ef None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:51.206898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860b1e9-9ae1-485e-8087-fd30013e87ef None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:51.207172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860b1e9-9ae1-485e-8087-fd30013e87ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:51.234512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4860b1e9-9ae1-485e-8087-fd30013e87ef None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:51.235710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4860b1e9-9ae1-485e-8087-fd30013e87ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:51.310429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 167/338] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:51.482947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e91d8-7e53-4444-9a34-92b120c5c78c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:51.541004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503cde1-6bcb-417a-bb22-f38c5121573a None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:51.541004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503cde1-6bcb-417a-bb22-f38c5121573a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:51.541004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503cde1-6bcb-417a-bb22-f38c5121573a None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:51.541736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 168/339] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:51.546295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30df63b-eb70-4932-81d8-5311d66c3000 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:51.546841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30df63b-eb70-4932-81d8-5311d66c3000 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:51.547318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30df63b-eb70-4932-81d8-5311d66c3000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:51.579565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e30df63b-eb70-4932-81d8-5311d66c3000 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:51.581133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e30df63b-eb70-4932-81d8-5311d66c3000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:51.601193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:51.678222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e91d8-7e53-4444-9a34-92b120c5c78c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:51.696776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 169/340] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:51.698058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b6cb-224e-4e4d-b8a3-87c7160ec8af None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:51.698406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b6cb-224e-4e4d-b8a3-87c7160ec8af None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:51.698683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b6cb-224e-4e4d-b8a3-87c7160ec8af None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:51.707064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 170/341] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:51.715844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e71e25-3e6f-4564-a3ed-d3db2cedc5ce None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:51.716133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e71e25-3e6f-4564-a3ed-d3db2cedc5ce None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:51.716410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e71e25-3e6f-4564-a3ed-d3db2cedc5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:51.737409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:51.742974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:51.743658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:51.743904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:51.744175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:51.756033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b3e71e25-3e6f-4564-a3ed-d3db2cedc5ce None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:51.757258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b3e71e25-3e6f-4564-a3ed-d3db2cedc5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:51.760007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:51.765337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:51.766465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:51.767280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:51.771065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 172/342] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:51.777310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:51.913077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 171/343] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:51.913940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e71e25-3e6f-4564-a3ed-d3db2cedc5ce None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:51.935521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93b695-6d95-4cb4-b5fa-9ba6660b308d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:51.998125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:52.008757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:52.010797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.010797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.010797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.032716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:52.039192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:52.040082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:52.040860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:52.046935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 173/344] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:52.049685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:52.130465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e71e25-3e6f-4564-a3ed-d3db2cedc5ce None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:52.199326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:52.200366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.200858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.201329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.202380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:52.202954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:52.204532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:52.205877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1440df2-315b-4b1c-abe5-640bfe4c2340 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:52.207139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:52.219635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 172/345] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:52.221002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:52.254051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:52.255975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.257276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.257923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.260108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:52.261495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:52.265773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:52.283233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 174/346] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:52.285431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:52.360961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:52.392565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c38c5-6949-47a3-a6d3-1d3e5dc42552 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:52.434719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04336d0-21fa-48ba-8266-20c8e986c7a1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:52.445122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:52.446402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.447138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.447620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.461018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:52.466370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:52.467362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:52.468146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 173/347] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:52.468878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:52.485098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:52.486320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.486853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.487546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.537881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:52.590910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:52.590910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:52.602049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dd81e-501a-4994-9a32-d4a993430211 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:52.650938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b0d91-1645-406b-9e59-3f8cf4d79b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:52.651560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b0d91-1645-406b-9e59-3f8cf4d79b99 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.651842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b0d91-1645-406b-9e59-3f8cf4d79b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.652119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b0d91-1645-406b-9e59-3f8cf4d79b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.654609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:52.726063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 175/348] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:52.726964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:52.749648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b0d91-1645-406b-9e59-3f8cf4d79b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:52.751147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b0d91-1645-406b-9e59-3f8cf4d79b99 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:52.829065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:52.854012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd817f-6b82-43e2-b983-5869e13af599 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:52.907325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 174/349] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:52.909924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e610e3-a086-4e86-9029-d7eecc4264dd None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.910174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:52.910334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e610e3-a086-4e86-9029-d7eecc4264dd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.910502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e610e3-a086-4e86-9029-d7eecc4264dd None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.910870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:52.911146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:52.911415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:52.912094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 175/350] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:52.912815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e610e3-a086-4e86-9029-d7eecc4264dd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:52.927380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:52.932512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:52.933431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:52.934191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 176/351] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:52.934915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:52.949022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:53.059371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e610e3-a086-4e86-9029-d7eecc4264dd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.077201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ca7e4-b185-4762-a9c5-b5e211c59bf6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.105706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.106875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.106875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.106875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.108117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:53.108117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.109109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.119746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 176/352] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.120684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.122567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c22c4a-0066-43f7-a1c1-365eb12c9f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.123212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22c4a-0066-43f7-a1c1-365eb12c9f6b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.123466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22c4a-0066-43f7-a1c1-365eb12c9f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.123748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c22c4a-0066-43f7-a1c1-365eb12c9f6b None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.124838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22c4a-0066-43f7-a1c1-365eb12c9f6b None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.126306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c22c4a-0066-43f7-a1c1-365eb12c9f6b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.130646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 177/353] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:52 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.130646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35569246-f5f0-407c-827e-a7741b240d0f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.130646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35569246-f5f0-407c-827e-a7741b240d0f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.130646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35569246-f5f0-407c-827e-a7741b240d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.147432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:53.149317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-35569246-f5f0-407c-827e-a7741b240d0f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:53.150669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-35569246-f5f0-407c-827e-a7741b240d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:53.227272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 178/354] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:53.228355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35569246-f5f0-407c-827e-a7741b240d0f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.239431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:53.248099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629842-5727-436c-be00-9a3150bf9073 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.295643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a58a98-8471-4a28-9560-e38894c11f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.296576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a58a98-8471-4a28-9560-e38894c11f5a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.296860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a58a98-8471-4a28-9560-e38894c11f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.297138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a58a98-8471-4a28-9560-e38894c11f5a None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.298311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a58a98-8471-4a28-9560-e38894c11f5a None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.301113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a58a98-8471-4a28-9560-e38894c11f5a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.303171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 177/355] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.307497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35017-7e45-4e23-a4a0-df612ba2a340 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.307777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35017-7e45-4e23-a4a0-df612ba2a340 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.308048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e35017-7e45-4e23-a4a0-df612ba2a340 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.309424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 178/356] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:53.309978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35017-7e45-4e23-a4a0-df612ba2a340 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.362854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35569246-f5f0-407c-827e-a7741b240d0f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.401442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.402435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.402929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.403973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.404644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:53.404944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.406781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.417324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:53.419127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 179/357] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.420090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.444300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e35017-7e45-4e23-a4a0-df612ba2a340 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.486054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.486793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.487440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.487559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] PATH_INFO: `/v3/users/0f70eec8436d4e2fa55791f2951e86fd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.488621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] RBAC: Authorizing `identity:get_user(user_id=0f70eec8436d4e2fa55791f2951e86fd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.490058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.491824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 179/358] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/users/0f70eec8436d4e2fa55791f2951e86fd => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.496889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.518461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:53.556988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b5cea-63e8-425e-bb98-67e08561aba6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.598360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4fad1-3bf1-4249-bd07-00f58f42552f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.599418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4fad1-3bf1-4249-bd07-00f58f42552f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.599418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4fad1-3bf1-4249-bd07-00f58f42552f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.599589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4fad1-3bf1-4249-bd07-00f58f42552f None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.609203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4fad1-3bf1-4249-bd07-00f58f42552f None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.610553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4fad1-3bf1-4249-bd07-00f58f42552f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.612291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 180/359] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.613381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ddc53-41e6-4dd8-a932-b8c5a488226e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.613671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ddc53-41e6-4dd8-a932-b8c5a488226e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.613945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ddc53-41e6-4dd8-a932-b8c5a488226e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.624495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd75d3-52df-4a6a-809d-c6adff78ff59 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.633945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f83ddc53-41e6-4dd8-a932-b8c5a488226e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:53.635179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f83ddc53-41e6-4dd8-a932-b8c5a488226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:53.666009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.666819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.667101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.667376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.668560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.669998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.671611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 180/360] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.672385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.698411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:53.714121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 181/361] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:53.714861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ddc53-41e6-4dd8-a932-b8c5a488226e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.797031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:53.806994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a59dd9-cf9a-4f96-ba16-c2e0272a65ff None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.848191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.848803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.849088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.849363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.849722 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ddc53-41e6-4dd8-a932-b8c5a488226e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:53.900730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:53.900730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:53.901046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:53.901046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:53.901765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] RBAC: Adding query filter params (role.id=d2c9d0ae0229447bb2c52d38aa7e005c, user.id=0f70eec8436d4e2fa55791f2951e86fd, scope.system=all) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:53.902068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.903524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.916700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 182/362] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/role_assignments?user.id=0f70eec8436d4e2fa55791f2951e86fd&scope.system=all&role.id=d2c9d0ae0229447bb2c52d38aa7e005c => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:53.922079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:53.958372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:53.958741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:53.988599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:54.070901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af9c5f-1770-4ccc-a350-f502c54c79fb None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.102377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:54.128285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 181/363] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:54.128880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.129161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.130669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.131114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.131510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.145305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:54.151499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:54.152390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:54.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 183/364] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:54.153843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.318202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456d604-7928-472b-b4b2-40cc292b6535 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.326127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59520c-8de1-4a21-82ff-25f0c54d81c5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.356785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:54.363288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.363998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.364272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.364540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.366264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=70b446c3c13d425d832af70478f512dd) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:54.366264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:54.366994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:54.371425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.371425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.371425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.371425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.371835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=70b446c3c13d425d832af70478f512dd) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:54.371835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:54.374175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:54.374309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 184/365] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:54.374309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.390545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 182/366] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:54.390545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.434739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:54.512854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3286c-7ef6-427e-90d7-f8e2aeeaaf6c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.543522 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f923a-5f9a-4c98-91f2-3db0bf4649fc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.566757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.567480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.567816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.568091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.583965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:54.587652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.588297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.588577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.588855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:54.589450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:54.590510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:54.591478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:54.592223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:54.592223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:54.592366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 185/367] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:54.597203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.597346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 183/368] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:54.598297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.673445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:54.732063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb037d4-a9c2-4b62-9e83-9dab1570cd96 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.744226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9874910-819d-4dee-9e66-38055757c723 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.747488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:54.773215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.774206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.774732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.775210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.776119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:54.776615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:54.778214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:54.789601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.790246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.790541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.790821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.791490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:54.791791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:54.793174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:54.794878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 184/369] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:54.796035 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.814257 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 186/370] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:54.814257 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:54.940225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48ac00-6f47-4b9d-848d-9f5e133d2823 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.964504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ba168-320e-4812-8667-fc4b2e2f0bd9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:54.967589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:54.988572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:54.989200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:54.989488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:54.989791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:54.990937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:54.993701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:54.995411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 185/371] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:54.996718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:55.007158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:55.008209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:55.008661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:55.009096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:55.046518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:55.094196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:55.097668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:55.119171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fa0a9-4796-40f9-b6a3-043e082b6ecf None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:55.166118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:55.166774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:55.167060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:55.167389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:55.212307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 187/372] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:55.213765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:55.248158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:55.265627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:55.267114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:55.290891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:55.329818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c71c93-b55a-459f-be92-eae7b29ae74e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:55.366821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:55.367750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:55.368244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:55.368855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:55.369905 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:55.370454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:55.372464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:55.379305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 188/373] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:45:55 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:55.380433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:55.406552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 186/374] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:45:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:45:55.413194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:55.493575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe91a89-5ec5-4b2b-a89c-911c9b512c86 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:55.529065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:55.534835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:55.535837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:55.536321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:55.536826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] PATH_INFO: `/v3/projects/ce1b87adbe93482a946d0fcf7c513809` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:55.538500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] RBAC: Authorizing `identity:get_project(project_id=ce1b87adbe93482a946d0fcf7c513809)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:55.540233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:55.546431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 189/375] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:55 2022] GET /identity/v3/projects/ce1b87adbe93482a946d0fcf7c513809 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:55.548261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:55.626115 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:55.644158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19e665-450b-49fb-9146-07886f843701 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:55.669760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecdfea-99a4-4324-9f44-b1232645beab None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:55.689626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:55.690687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:55.691170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:55.691643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:55.693467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:55.695300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:55.697645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 187/376] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:55 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:55.702771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:55.754763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:55.760293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:55.760818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:55.761309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] PATH_INFO: `/v3/groups/f22efcd3449445bb80c1f44a64d92bd2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:55.769767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] RBAC: Authorizing `identity:get_group(group_id=f22efcd3449445bb80c1f44a64d92bd2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:55.772254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:55.774833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 190/377] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:55 2022] GET /identity/v3/groups/f22efcd3449445bb80c1f44a64d92bd2 => generated 229 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:55.788842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:55.890161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5625fd-f3aa-4728-be38-fce20235a33a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:55.904504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:55.969504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:55.970145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:55.970451 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:55.970788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:55.972754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:55.974223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:55.978202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 188/378] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:55 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:55.981048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a036b-acdf-42fe-8390-d4cdc01103be None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:55.983897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:56.015590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:56.051696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f4080-84ee-4b02-b2ff-8194df9c5d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:56.052248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4080-84ee-4b02-b2ff-8194df9c5d71 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:56.053230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4080-84ee-4b02-b2ff-8194df9c5d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:56.053502 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4080-84ee-4b02-b2ff-8194df9c5d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:56.054315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f4080-84ee-4b02-b2ff-8194df9c5d71 None admin] RBAC: Adding query filter params (group.id=f22efcd3449445bb80c1f44a64d92bd2, role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=ce1b87adbe93482a946d0fcf7c513809) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:56.054621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f4080-84ee-4b02-b2ff-8194df9c5d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:56.056347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f4080-84ee-4b02-b2ff-8194df9c5d71 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:56.071147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 191/379] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:45:55 2022] GET /identity/v3/role_assignments?group.id=f22efcd3449445bb80c1f44a64d92bd2&scope.project.id=ce1b87adbe93482a946d0fcf7c513809&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 260 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:56.106012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681c5f5-fd96-4bda-989a-4c579bac4b30 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:56.145531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a8c90-c31a-4f20-85c6-d7d1c6af60b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:56.146144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8c90-c31a-4f20-85c6-d7d1c6af60b3 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:56.146455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8c90-c31a-4f20-85c6-d7d1c6af60b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:56.146731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8c90-c31a-4f20-85c6-d7d1c6af60b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:56.147474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a8c90-c31a-4f20-85c6-d7d1c6af60b3 None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:56.147795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a8c90-c31a-4f20-85c6-d7d1c6af60b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:56.149221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a8c90-c31a-4f20-85c6-d7d1c6af60b3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:56.155476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 189/380] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:45:55 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:56.669442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef8ff-1ab5-457e-9416-1250866c9be2 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:56.669947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef8ff-1ab5-457e-9416-1250866c9be2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:56.670019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcef8ff-1ab5-457e-9416-1250866c9be2 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:56.671499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 192/381] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:56.677460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be604806-383f-4a80-932b-47943a194cc7 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:56.677967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be604806-383f-4a80-932b-47943a194cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:56.677967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be604806-383f-4a80-932b-47943a194cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:56.703380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-be604806-383f-4a80-932b-47943a194cc7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:56.704612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-be604806-383f-4a80-932b-47943a194cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:56.723830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e0020-3ef4-4e25-9629-bf020c1d261e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:56.724121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e0020-3ef4-4e25-9629-bf020c1d261e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:56.724370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e0020-3ef4-4e25-9629-bf020c1d261e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:56.725697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 193/382] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:45:56.741962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd0f4-108d-4226-b38d-fe971979bade None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:56.742562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd0f4-108d-4226-b38d-fe971979bade None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:56.743006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd0f4-108d-4226-b38d-fe971979bade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:56.774774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f1dcd0f4-108d-4226-b38d-fe971979bade None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:56.776358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f1dcd0f4-108d-4226-b38d-fe971979bade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:56.787656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 190/383] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:56.891756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 194/384] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:57.195634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be604806-383f-4a80-932b-47943a194cc7 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:57.211722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:57.249422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd0f4-108d-4226-b38d-fe971979bade None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:57.338020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be604806-383f-4a80-932b-47943a194cc7 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:57.358579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:57.407173 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:57.407811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:57.408147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:57.408432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:57.429083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:57.433823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:57.434675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:57.435438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 191/385] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:57.441621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:57.441750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dcd0f4-108d-4226-b38d-fe971979bade None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:57.523103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:57.523764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:57.524043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:57.524315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:57.548065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:57.553643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:57.554677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:57.555459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:57.563436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 195/386] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:45:57.569099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:57.586583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:57.638890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b460-7b10-4bb0-94ae-e63910a175e7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:57.688894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816b2d-c6cd-45fb-9f0c-b4f3d0667bd1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:57.691281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818226ec-56c4-4b85-ab11-59219232a455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:57.693112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818226ec-56c4-4b85-ab11-59219232a455 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:57.693535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818226ec-56c4-4b85-ab11-59219232a455 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:57.693946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818226ec-56c4-4b85-ab11-59219232a455 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:57.694773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818226ec-56c4-4b85-ab11-59219232a455 None admin] RBAC: Adding query filter params (name=member) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:57.695229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818226ec-56c4-4b85-ab11-59219232a455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:57.696726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818226ec-56c4-4b85-ab11-59219232a455 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:57.707185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:57.709961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 192/387] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:57.711802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc4c3a-57a2-4e7e-87e9-896307e2ed59 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:57.712254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc4c3a-57a2-4e7e-87e9-896307e2ed59 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:57.712933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc4c3a-57a2-4e7e-87e9-896307e2ed59 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:57.714513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 193/388] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:57.715644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc4c3a-57a2-4e7e-87e9-896307e2ed59 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:57.730545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f05565-cf34-4a2b-adf8-0db5642ea9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:57.731463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f05565-cf34-4a2b-adf8-0db5642ea9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:57.732296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f05565-cf34-4a2b-adf8-0db5642ea9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:57.732584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f05565-cf34-4a2b-adf8-0db5642ea9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:57.733397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f05565-cf34-4a2b-adf8-0db5642ea9c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:57.734117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f05565-cf34-4a2b-adf8-0db5642ea9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:57.736114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f05565-cf34-4a2b-adf8-0db5642ea9c3 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:57.745272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 196/389] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:57.750047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c7e7c-535b-4346-89eb-e9a9cec02a17 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:57.750047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c7e7c-535b-4346-89eb-e9a9cec02a17 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:57.750047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c7e7c-535b-4346-89eb-e9a9cec02a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:57.767742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-115c7e7c-535b-4346-89eb-e9a9cec02a17 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:57.769084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-115c7e7c-535b-4346-89eb-e9a9cec02a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:57.850476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 197/390] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:57.851506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c7e7c-535b-4346-89eb-e9a9cec02a17 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:57.853112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc4c3a-57a2-4e7e-87e9-896307e2ed59 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:57.894442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:57.895311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:57.895731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:57.896393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:57.898025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:57.899658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:57.902509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 194/391] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:57 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:57.910263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:57.914256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:58.017551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d05c15-af06-4520-9f3e-52fe5432a45a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.054474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:58.055109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.055389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.055664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.057860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:58.059335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:58.067060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 195/392] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:57 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:58.071872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:58.074047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c7e7c-535b-4346-89eb-e9a9cec02a17 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.088597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:58.139371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:58.139935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.140214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.140483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.147530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:58.148996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:58.150663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 198/393] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:57 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:58.156437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:58.215283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:58.248678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4c3a1-73da-418d-a6d3-ef2dd88b7345 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.288644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:58.289581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.289788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53eff6-8757-4e13-bcc9-aee87dff8705 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.290164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.290601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.292929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] RBAC: Authorizing `identity:create_grant(project_id=70b446c3c13d425d832af70478f512dd, group_id=1c67857e45b245f3957f0ecdd9f07ff0, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:58.294528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:58.326218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:58.326891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.327168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.327434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.328558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:58.330865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:58.332691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 199/394] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:58 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:58.333369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:58.359655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 196/395] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 22 10:45:58 2022] PUT /identity/v3/projects/70b446c3c13d425d832af70478f512dd/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:58.361138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:58.393880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:58.489990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4cc3b-595a-46b1-b9a2-99431ae7f0d5 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.525016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadbb2f-2ea5-4011-9228-23cf4b44229a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.529812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1de15-abf6-4ff1-8767-c85bb9b7967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:58.532014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1de15-abf6-4ff1-8767-c85bb9b7967f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.532014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1de15-abf6-4ff1-8767-c85bb9b7967f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.532014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1de15-abf6-4ff1-8767-c85bb9b7967f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.543623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1de15-abf6-4ff1-8767-c85bb9b7967f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-5be1de15-abf6-4ff1-8767-c85bb9b7967f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:58.548712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:58.549597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:45:58.550395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 200/396] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:58.551849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954e56-9711-46d5-8bbe-5e79f02bc129 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.551849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954e56-9711-46d5-8bbe-5e79f02bc129 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.553411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12954e56-9711-46d5-8bbe-5e79f02bc129 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.553411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 201/397] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:58.554399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e28ee5-be08-4d6c-84aa-f3e2a07ea06c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.554802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e28ee5-be08-4d6c-84aa-f3e2a07ea06c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.555106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e28ee5-be08-4d6c-84aa-f3e2a07ea06c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.556415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 202/398] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:45:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:45:58.561565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6888ecc-2397-46bd-8b42-ccb5cfe04295 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.561565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6888ecc-2397-46bd-8b42-ccb5cfe04295 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.561565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6888ecc-2397-46bd-8b42-ccb5cfe04295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.566275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:58.573460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6de5c8-bbb7-4bf8-95d9-31422a1930f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:58.574127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6de5c8-bbb7-4bf8-95d9-31422a1930f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.574513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6de5c8-bbb7-4bf8-95d9-31422a1930f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.574789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6de5c8-bbb7-4bf8-95d9-31422a1930f6 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.576969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6de5c8-bbb7-4bf8-95d9-31422a1930f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=70b446c3c13d425d832af70478f512dd, group_id=1c67857e45b245f3957f0ecdd9f07ff0, role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:58.578475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6de5c8-bbb7-4bf8-95d9-31422a1930f6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:58.583397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f6888ecc-2397-46bd-8b42-ccb5cfe04295 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:58.583397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f6888ecc-2397-46bd-8b42-ccb5cfe04295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:58.648892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 203/399] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:45:58.651169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6888ecc-2397-46bd-8b42-ccb5cfe04295 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:58.659892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 197/400] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 22 10:45:58 2022] PUT /identity/v3/projects/70b446c3c13d425d832af70478f512dd/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:58.666163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acb1b4-f3c9-4c74-958d-eee1c902e7dd None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.669074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acb1b4-f3c9-4c74-958d-eee1c902e7dd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.671893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acb1b4-f3c9-4c74-958d-eee1c902e7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.694383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:58.720466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-33acb1b4-f3c9-4c74-958d-eee1c902e7dd None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:45:58.721874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-33acb1b4-f3c9-4c74-958d-eee1c902e7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:45:58.770817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6888ecc-2397-46bd-8b42-ccb5cfe04295 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.841941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:58.842607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:58.842889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:58.843162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:58.843837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:58.844140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:58.845561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:58.855725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 204/401] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:45:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:58.861549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:58.875102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 198/402] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:45:58.942169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acb1b4-f3c9-4c74-958d-eee1c902e7dd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:58.983378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275cab2-1308-4252-833f-c0daf55704bf None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:58.987983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:59.060082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.060733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.061009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.061278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.062674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:59.064112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:59.067157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 205/403] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:58 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:45:59.073172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.100719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acb1b4-f3c9-4c74-958d-eee1c902e7dd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.148906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:59.190953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.191609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.191895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.192227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.195560 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f993f-488a-430d-abce-20d6791779c2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.216353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:59.222516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:59.223509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:45:59.224356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:59.226960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 199/404] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:45:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:59.228013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.237442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.238261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.238368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.238641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.239781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:59.241218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:59.243185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 206/405] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:59 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:59.244012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.277800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:59.338798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7569112-a65a-46ff-9a22-5367c8599f21 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.348906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc33e7e-b07f-4aa6-8ea4-10457dccb466 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.375889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.377354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.377826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.378312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.384076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.384735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.385021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.385296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.385953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:59.386422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:59.387654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:59.391490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:45:59.396600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 207/406] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:45:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:45:59.397357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:45:59.398199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:45:59.398985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:45:59.401425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 200/407] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:45:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:45:59.406351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.406494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.454955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:59.515818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34fc03-a709-440d-950a-f231d3b8e904 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.534710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:59.544061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f507e-2eb7-445e-911a-55751c18d7b8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.554994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.556261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.556763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.557233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.558749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:59.560645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:59.566654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 201/408] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:59 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:59.567934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.587900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.588572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.588855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.589357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.591687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] RBAC: Authorizing `identity:create_grant(project_id=d8c6befab82a430b8b4f72131c2e9e57, group_id=1c67857e45b245f3957f0ecdd9f07ff0, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:59.593166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:59.672468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 208/409] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 22 10:45:59 2022] PUT /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:45:59.674263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.678338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e6ba-68b6-47a8-be36-63282d2ced4c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.691856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:59.725140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.726938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.727443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.727910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.728835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:45:59.729343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:59.731189 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:59.742472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 202/410] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:45:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:59.748536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.810927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff35f6-9866-46bc-9c2f-9083daf4ff3b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:45:59.855378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:45:59.856044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:45:59.856308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:45:59.856579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:45:59.857725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:45:59.859204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:45:59.862048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 209/411] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:45:59 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:45:59.868398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:45:59.905005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:45:59.933076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248a1e-67a0-4e1c-861a-c13c9cdde863 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:00.002278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:00.003261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.003686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.004109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] PATH_INFO: `/v3/groups/f22efcd3449445bb80c1f44a64d92bd2` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.006975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] RBAC: Authorizing `identity:get_group(group_id=f22efcd3449445bb80c1f44a64d92bd2)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:00.008579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:00.012433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 203/412] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:45:59 2022] GET /identity/v3/groups/f22efcd3449445bb80c1f44a64d92bd2 => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:00.018852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:00.035071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7e13c-d9f3-4813-bb80-03a1702eea40 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:00.070877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:00.089676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:00.090383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.090658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.090928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.097035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] RBAC: Authorizing `identity:create_grant(project_id=d8c6befab82a430b8b4f72131c2e9e57, group_id=1c67857e45b245f3957f0ecdd9f07ff0, role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:00.098522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:00.145540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585be91-f64f-4eb6-b83d-c1824500dd8e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:00.167781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 210/413] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 22 10:45:59 2022] PUT /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57/groups/1c67857e45b245f3957f0ecdd9f07ff0/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:00.191320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c996d5a-2847-4ddb-9e2f-7f030e8e25dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:00.191975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996d5a-2847-4ddb-9e2f-7f030e8e25dd None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.192251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996d5a-2847-4ddb-9e2f-7f030e8e25dd None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.192523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996d5a-2847-4ddb-9e2f-7f030e8e25dd None admin] PATH_INFO: `/v3/projects/ce1b87adbe93482a946d0fcf7c513809` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.193729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c996d5a-2847-4ddb-9e2f-7f030e8e25dd None admin] RBAC: Authorizing `identity:get_project(project_id=ce1b87adbe93482a946d0fcf7c513809)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:00.195429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c996d5a-2847-4ddb-9e2f-7f030e8e25dd None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:00.197768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 204/414] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:46:00 2022] GET /identity/v3/projects/ce1b87adbe93482a946d0fcf7c513809 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:00.204018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:00.306262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:00.327830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cd627-02ae-4f73-aa0e-7a52e09c73ad None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:00.374507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe06dd-53f7-414f-9987-e27493255483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:00.375214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe06dd-53f7-414f-9987-e27493255483 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.375578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe06dd-53f7-414f-9987-e27493255483 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.375847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe06dd-53f7-414f-9987-e27493255483 None admin] PATH_INFO: `/v3/projects/ce1b87adbe93482a946d0fcf7c513809/groups/f22efcd3449445bb80c1f44a64d92bd2/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.378713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe06dd-53f7-414f-9987-e27493255483 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1b87adbe93482a946d0fcf7c513809, group_id=f22efcd3449445bb80c1f44a64d92bd2, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:00.380204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe06dd-53f7-414f-9987-e27493255483 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:00.464879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 211/415] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 22 10:46:00 2022] PUT /identity/v3/projects/ce1b87adbe93482a946d0fcf7c513809/groups/f22efcd3449445bb80c1f44a64d92bd2/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:00.605195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716c487-5d32-41cf-80ec-55050873a940 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.605385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716c487-5d32-41cf-80ec-55050873a940 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.605757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716c487-5d32-41cf-80ec-55050873a940 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.607432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 205/416] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:00.613650 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1849-8a28-4ec8-a419-5c4774bdd95a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.613931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1849-8a28-4ec8-a419-5c4774bdd95a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.614217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1849-8a28-4ec8-a419-5c4774bdd95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.639815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-dddf1849-8a28-4ec8-a419-5c4774bdd95a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:00.641012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dddf1849-8a28-4ec8-a419-5c4774bdd95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:00.714191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 212/417] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:00.923185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295691f5-602f-479c-8fe6-00ad3af861e5 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.924475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295691f5-602f-479c-8fe6-00ad3af861e5 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.925129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295691f5-602f-479c-8fe6-00ad3af861e5 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.927017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 206/418] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:00.933383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65913-a91e-4f8d-b63a-c6ef56ce5ea5 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:00.934024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65913-a91e-4f8d-b63a-c6ef56ce5ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:00.934024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65913-a91e-4f8d-b63a-c6ef56ce5ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:00.959293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-8db65913-a91e-4f8d-b63a-c6ef56ce5ea5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:00.960508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8db65913-a91e-4f8d-b63a-c6ef56ce5ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:01.051386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 213/419] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:01.116994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295691f5-602f-479c-8fe6-00ad3af861e5 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:01.252430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295691f5-602f-479c-8fe6-00ad3af861e5 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:01.261601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:01.291836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:01.292481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:01.292762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:01.293046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:01.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:01.317149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:01.318005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:46:01.318813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:01.322568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 207/420] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:01.326061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65913-a91e-4f8d-b63a-c6ef56ce5ea5 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:01.328629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:01.383607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:01.478472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65913-a91e-4f8d-b63a-c6ef56ce5ea5 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:01.516648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:01.518082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:01.518485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:01.518762 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:01.519438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:01.519742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:01.521317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:01.531064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 214/421] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:46:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:01.539489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:01.624305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdf367-6813-4485-9db6-b9a4ca925967 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:01.634832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:01.683279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:01.683918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:01.684196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:01.684465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:01.704708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:01.710149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:01.711275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:46:01.712251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:01.715192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522c1f2-e99b-4138-b61e-a8fabc4453d2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:01.724553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 208/422] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:01.725472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:01.778931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:01.779564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:01.779840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:01.780107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:01.782608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:01.909472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:01.910881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:01.928497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac1593-d7ab-450f-9d06-699d4b83fdc4 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:01.996532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:01.997139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:01.997416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:01.997682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:01.998365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:01.998661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:02.000009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:02.021957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 209/423] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:02.030978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:02.035771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:02.083175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 215/424] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:02.089244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:02.205977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ece77-5576-4839-8c33-69a9ca207841 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:02.207260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:02.217254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abca143-ffc5-46ac-a0bc-9066dd665cb6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:02.243541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357951a1-1125-4bd7-89a1-9a15c8588b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:02.244277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357951a1-1125-4bd7-89a1-9a15c8588b1a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.244564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357951a1-1125-4bd7-89a1-9a15c8588b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.244836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357951a1-1125-4bd7-89a1-9a15c8588b1a None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.246356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357951a1-1125-4bd7-89a1-9a15c8588b1a None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:02.247900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357951a1-1125-4bd7-89a1-9a15c8588b1a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:02.249992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 216/425] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:02.251176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567619f1-22b7-438f-930c-63bfa05180e6 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.251465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567619f1-22b7-438f-930c-63bfa05180e6 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.252366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567619f1-22b7-438f-930c-63bfa05180e6 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.253111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 217/426] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:02.258582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567619f1-22b7-438f-930c-63bfa05180e6 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:02.258862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:02.259582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.259928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.260268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.292911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:02.335195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:02.336953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:02.373739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567619f1-22b7-438f-930c-63bfa05180e6 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:02.411893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83c86f-6e4a-4c9c-8408-f0e529f41e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:02.412962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83c86f-6e4a-4c9c-8408-f0e529f41e53 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.413443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83c86f-6e4a-4c9c-8408-f0e529f41e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.413912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83c86f-6e4a-4c9c-8408-f0e529f41e53 None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.415302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83c86f-6e4a-4c9c-8408-f0e529f41e53 None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:02.417052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83c86f-6e4a-4c9c-8408-f0e529f41e53 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:02.418855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 218/427] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:02.420280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fbf52-6b0f-4abd-9b76-a57277f1ba26 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.420790 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fbf52-6b0f-4abd-9b76-a57277f1ba26 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.421327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fbf52-6b0f-4abd-9b76-a57277f1ba26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.434195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:02.441794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4f1fbf52-6b0f-4abd-9b76-a57277f1ba26 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:02.443295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4f1fbf52-6b0f-4abd-9b76-a57277f1ba26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:02.449184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 210/428] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 22 10:46:02.449184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:02.522400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 219/429] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:02.530251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fbf52-6b0f-4abd-9b76-a57277f1ba26 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:02.556153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45822939-ddcc-4cd4-82ca-0caa85ec2444 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:02.580120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:02.597311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474bc9b-84de-4bb6-8672-d6a77027fe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:02.598273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474bc9b-84de-4bb6-8672-d6a77027fe69 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.598730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474bc9b-84de-4bb6-8672-d6a77027fe69 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.599226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474bc9b-84de-4bb6-8672-d6a77027fe69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.600158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474bc9b-84de-4bb6-8672-d6a77027fe69 None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=70b446c3c13d425d832af70478f512dd) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:02.600644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474bc9b-84de-4bb6-8672-d6a77027fe69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:02.602371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474bc9b-84de-4bb6-8672-d6a77027fe69 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:02.608859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 211/430] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:02.610316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09815f-777b-418d-9a55-98ab0e8abee8 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.610805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09815f-777b-418d-9a55-98ab0e8abee8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.612284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09815f-777b-418d-9a55-98ab0e8abee8 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.614590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 212/431] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:02.619938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bbb6b-bb91-4588-a288-71260db91ffe None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.620512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bbb6b-bb91-4588-a288-71260db91ffe None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.621394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bbb6b-bb91-4588-a288-71260db91ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.642104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e71bbb6b-bb91-4588-a288-71260db91ffe None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:02.643839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e71bbb6b-bb91-4588-a288-71260db91ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:02.661601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fbf52-6b0f-4abd-9b76-a57277f1ba26 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:02.708055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:02.708655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:02.709342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.709829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.710441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] PATH_INFO: `/v3/projects/70b446c3c13d425d832af70478f512dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.715756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] RBAC: Authorizing `identity:get_project(project_id=70b446c3c13d425d832af70478f512dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:02.717655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:02.721285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 220/432] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/projects/70b446c3c13d425d832af70478f512dd => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:02.726896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:02.741658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 213/433] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:02.797495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35a33f-c547-439a-83ec-d2b01a485e7e None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.797773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35a33f-c547-439a-83ec-d2b01a485e7e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.797974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35a33f-c547-439a-83ec-d2b01a485e7e None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.800552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 214/434] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:02.807597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864a0a-b56b-41d3-a0cb-4e746721522c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.807597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864a0a-b56b-41d3-a0cb-4e746721522c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.808104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31864a0a-b56b-41d3-a0cb-4e746721522c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.831028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-31864a0a-b56b-41d3-a0cb-4e746721522c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:02.832416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-31864a0a-b56b-41d3-a0cb-4e746721522c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:02.882586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:02.887848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edc63c-7ca6-4722-88b6-107b75dc3002 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:02.929577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 215/435] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:02.929577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864a0a-b56b-41d3-a0cb-4e746721522c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:02.937211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:02.938114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:02.939172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:02.939891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:02.942770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:02.946335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:02.947620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 221/436] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:02.953516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.067707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31864a0a-b56b-41d3-a0cb-4e746721522c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.084424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.093202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4313d-e8e3-4ab6-b6c7-b9972dc5dd55 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.108788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.109429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.109706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.109976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.122738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:03.128057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:03.128920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 22 10:46:03.129756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:03.133853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.135725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.135832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 216/437] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:03.135922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.135922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.136044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.136109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=70b446c3c13d425d832af70478f512dd) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:03.136109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:03.137406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:03.144853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 222/438] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:46:02 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=70b446c3c13d425d832af70478f512dd&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:03.145430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.182649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.271879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517617-3ba5-48ac-b5cf-be0d9f094971 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.286644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db4000-198a-484e-afd4-2b536858cfd2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.321424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.322322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.322831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.323364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.331347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.331981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.332262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.332538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.333210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:03.333522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:03.335062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:03.337876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:03.342810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:03.343661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 217/439] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:03.344829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.348491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 223/440] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:03.352528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.389034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.460238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.465713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc238b-4638-47c5-be0b-7d7d5965dea6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.470132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4c2d-e7f7-4f3a-88d7-079ce83804ec None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.506216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.506920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.507045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.507257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.507474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.507575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.507860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.508132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.508802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:03.509168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:03.510600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:03.519360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:03.523678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 224/441] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:03.526755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:03.527581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:03.528365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:03.529000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.529068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 218/442] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:03.529068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.627075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.631550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180cef-a89d-4050-a809-422aadd0aae6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.633754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d48e4c-b11b-47a2-bc01-7d0a08b5afaa None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.664430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.665037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.665322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.665593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.666356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.666965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.667249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.667523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.689810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.733179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:03.733659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:03.734662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:03.735192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:03.825305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.832557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 225/443] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:03.833477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.834388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 219/444] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:03.838210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.892482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:03.937127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138cbbc-d1bf-42a2-8890-ade2678a6572 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.937400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c770c-13c9-4b55-ae4a-7e2f757f1cd7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:03.969485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.969976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:03.970115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.971052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.972141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.972242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:03.972242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:03.972242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:03.972454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:03.972454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:03.972943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:03.973564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:03.974404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:03.976026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 220/445] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:03.976778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:03.981822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 226/446] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:03.982524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.037514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:04.078416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90894d-56ae-4c17-8c15-852dd93c68b4 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.081360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c1997-1073-45f6-a040-9fe1add5a87d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.096953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:04.114402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.114846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.115003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.115285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.115555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] PATH_INFO: `/v3/projects/d8c6befab82a430b8b4f72131c2e9e57` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.115652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.116155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.116430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.116647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] RBAC: Authorizing `identity:get_project(project_id=d8c6befab82a430b8b4f72131c2e9e57)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.117542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.118058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.118992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.119703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 221/447] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/projects/d8c6befab82a430b8b4f72131c2e9e57 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:04.120453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.121107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 227/448] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:46:03 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:04.125340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.219468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83341b5-354a-4ee1-baa9-1efed9b36fff None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.223919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4b3f3-42bd-4354-8444-f6226f8a058d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.249073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:04.255020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.255616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.255892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.275766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.275892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.275892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.275892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.275892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] PATH_INFO: `/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.276200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] RBAC: Authorizing `identity:get_group(group_id=1c67857e45b245f3957f0ecdd9f07ff0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.276200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.276200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 228/449] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:46:04 2022] GET /identity/v3/groups/1c67857e45b245f3957f0ecdd9f07ff0 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:04.276200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.316822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:04.323842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.325306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.362437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145384f-179c-481e-97d4-03c59a52cd2a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.397091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.397697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.397973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.398268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.398982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:04.399283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.400713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.406274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 229/450] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:46:04 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:04.407037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.453742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 222/451] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:04.455580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.501499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:04.531607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0356477b-3d74-42c7-b374-f53178dcce0b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.551358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:04.572068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.572256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.572465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.576682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.576825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d558da-cce9-426a-8ef4-ff37d77dd845 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.576906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] RBAC: Adding query filter params (group.id=1c67857e45b245f3957f0ecdd9f07ff0, role.id=f4ba3d348adb4b6497d8a69e88e3822b, scope.project.id=d8c6befab82a430b8b4f72131c2e9e57) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:04.576906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.576906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.584417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 230/452] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:46:04 2022] GET /identity/v3/role_assignments?group.id=1c67857e45b245f3957f0ecdd9f07ff0&scope.project.id=d8c6befab82a430b8b4f72131c2e9e57&role.id=f4ba3d348adb4b6497d8a69e88e3822b => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:04.616537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.617185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.617466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.617742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] PATH_INFO: `/v3/projects/ce1b87adbe93482a946d0fcf7c513809` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.618966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] RBAC: Authorizing `identity:get_project(project_id=ce1b87adbe93482a946d0fcf7c513809)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.620432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.622402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 223/453] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:46:04 2022] GET /identity/v3/projects/ce1b87adbe93482a946d0fcf7c513809 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:04.628930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.737331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f432a94-2abd-4f28-a001-0eb7d2e85f6c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.775605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eadb4-4fcd-4d73-86aa-389de26e0697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.776166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eadb4-4fcd-4d73-86aa-389de26e0697 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.776442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eadb4-4fcd-4d73-86aa-389de26e0697 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.776707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eadb4-4fcd-4d73-86aa-389de26e0697 None admin] PATH_INFO: `/v3/groups/f22efcd3449445bb80c1f44a64d92bd2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.777804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eadb4-4fcd-4d73-86aa-389de26e0697 None admin] RBAC: Authorizing `identity:get_group(group_id=f22efcd3449445bb80c1f44a64d92bd2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.779293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eadb4-4fcd-4d73-86aa-389de26e0697 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.780840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 231/454] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 22 10:46:04 2022] GET /identity/v3/groups/f22efcd3449445bb80c1f44a64d92bd2 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:04.787352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:04.893449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:04.897335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bfc92-ff24-486e-b44e-6f1c13232399 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:04.932356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970ff2b-6146-42d8-8187-e497df3da2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:04.933019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970ff2b-6146-42d8-8187-e497df3da2ab None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:04.933300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970ff2b-6146-42d8-8187-e497df3da2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:04.933569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970ff2b-6146-42d8-8187-e497df3da2ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:04.934328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970ff2b-6146-42d8-8187-e497df3da2ab None admin] RBAC: Adding query filter params (group.id=f22efcd3449445bb80c1f44a64d92bd2, role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=ce1b87adbe93482a946d0fcf7c513809) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:04.934616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970ff2b-6146-42d8-8187-e497df3da2ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:04.936028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970ff2b-6146-42d8-8187-e497df3da2ab None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:04.943771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 224/455] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 22 10:46:04 2022] GET /identity/v3/role_assignments?group.id=f22efcd3449445bb80c1f44a64d92bd2&scope.project.id=ce1b87adbe93482a946d0fcf7c513809&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:07.231535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673066d5-1774-4524-bfba-eb4e0beef022 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:07.232411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673066d5-1774-4524-bfba-eb4e0beef022 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:07.232838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673066d5-1774-4524-bfba-eb4e0beef022 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:07.234597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 232/456] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:07.240921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878404-0dbd-4d5e-96d6-10988c7ade65 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:07.241709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878404-0dbd-4d5e-96d6-10988c7ade65 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:07.242139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50878404-0dbd-4d5e-96d6-10988c7ade65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:07.266128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-50878404-0dbd-4d5e-96d6-10988c7ade65 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:07.268119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-50878404-0dbd-4d5e-96d6-10988c7ade65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:07.357477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 225/457] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:07.384970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a6090-a3d3-4a9d-954f-153b1bd3f2c8 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:07.385500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a6090-a3d3-4a9d-954f-153b1bd3f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:07.385914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a6090-a3d3-4a9d-954f-153b1bd3f2c8 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:07.387422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 233/458] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:07.393654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84497b92-6a8a-413d-b652-1e3b7650d01e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:07.394427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84497b92-6a8a-413d-b652-1e3b7650d01e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:07.395153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84497b92-6a8a-413d-b652-1e3b7650d01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:07.418604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-84497b92-6a8a-413d-b652-1e3b7650d01e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:07.420135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-84497b92-6a8a-413d-b652-1e3b7650d01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:07.510929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 226/459] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:07.754768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a6090-a3d3-4a9d-954f-153b1bd3f2c8 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:07.832392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:07.875711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f835e0-bf34-4fa5-8433-944cacb018a0 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:07.876038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f835e0-bf34-4fa5-8433-944cacb018a0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:07.876340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f835e0-bf34-4fa5-8433-944cacb018a0 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:07.877980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 227/460] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:07.882767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9552749-f209-467d-8e33-ffb3065de073 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:07.883346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9552749-f209-467d-8e33-ffb3065de073 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:07.883833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9552749-f209-467d-8e33-ffb3065de073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:07.905769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-a9552749-f209-467d-8e33-ffb3065de073 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:07.907929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a9552749-f209-467d-8e33-ffb3065de073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:07.916659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a6090-a3d3-4a9d-954f-153b1bd3f2c8 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:07.926768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:07.959348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:07.959994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:07.960285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:07.960822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:07.974465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:07.976511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:07.986772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 228/461] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:07.988003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9552749-f209-467d-8e33-ffb3065de073 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:07.995163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:07.996380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:07.997301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:07.997301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:07.997301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:08.001906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 234/462] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:08.003194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:08.127082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:08.131262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6217b5-7d0c-4690-a3f1-a6dbad8530f8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:08.138639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9552749-f209-467d-8e33-ffb3065de073 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:08.172492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:08.173746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.174283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.174853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.175794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:08.176309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:08.178006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:08.184089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c37d66-cd34-4428-9d87-c4fdbdd455fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:08.184731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37d66-cd34-4428-9d87-c4fdbdd455fe None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.185017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37d66-cd34-4428-9d87-c4fdbdd455fe None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.185295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c37d66-cd34-4428-9d87-c4fdbdd455fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.197643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c37d66-cd34-4428-9d87-c4fdbdd455fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:08.198507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 235/463] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:08.203608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c37d66-cd34-4428-9d87-c4fdbdd455fe None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:08.216120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-c2c37d66-cd34-4428-9d87-c4fdbdd455fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:08.221985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:08.222885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:08.223777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:08.223777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:08.223777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:08.225637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 229/464] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:08.228151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51ef2-48f1-4f76-ba08-0312ece9ffe5 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.228673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51ef2-48f1-4f76-ba08-0312ece9ffe5 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.229160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51ef2-48f1-4f76-ba08-0312ece9ffe5 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.230807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 230/465] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:08.231762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51ef2-48f1-4f76-ba08-0312ece9ffe5 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:08.237158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:08.352572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea641-b5ea-4a0b-8916-a57184079a43 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:08.357711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51ef2-48f1-4f76-ba08-0312ece9ffe5 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:08.396546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:08.396950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89de75a-4e15-4ef3-970b-3c61da30fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:08.397074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.397363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.397550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de75a-4e15-4ef3-970b-3c61da30fa1f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.397662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.397883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de75a-4e15-4ef3-970b-3c61da30fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.398159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89de75a-4e15-4ef3-970b-3c61da30fa1f None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.398335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:08.398827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89de75a-4e15-4ef3-970b-3c61da30fa1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:08.399126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89de75a-4e15-4ef3-970b-3c61da30fa1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:08.399748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:08.400475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89de75a-4e15-4ef3-970b-3c61da30fa1f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:08.414485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 231/466] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:08.420168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d0da-260b-413b-9324-88b4a24a08c5 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.420455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d0da-260b-413b-9324-88b4a24a08c5 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.425529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396d0da-260b-413b-9324-88b4a24a08c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.441053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8396d0da-260b-413b-9324-88b4a24a08c5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:08.442732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8396d0da-260b-413b-9324-88b4a24a08c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:08.514265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 236/467] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:46:08 2022] POST /identity/v3/users => generated 246 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:08.514265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:08.521598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:08.531123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 232/468] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:08.531806 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396d0da-260b-413b-9324-88b4a24a08c5 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:08.558216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:08.705553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396d0da-260b-413b-9324-88b4a24a08c5 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:08.707088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a0bffb-1dfb-46bc-979f-20061beba2bf None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:08.742506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eba334-a208-4d3f-baca-96446ff48bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:08.743113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eba334-a208-4d3f-baca-96446ff48bc0 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.743387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eba334-a208-4d3f-baca-96446ff48bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.743522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f810c-2df6-409c-839a-290520d46934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:08.743641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eba334-a208-4d3f-baca-96446ff48bc0 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.744277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eba334-a208-4d3f-baca-96446ff48bc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:08.744379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f810c-2df6-409c-839a-290520d46934 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.744627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f810c-2df6-409c-839a-290520d46934 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.744967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f810c-2df6-409c-839a-290520d46934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.745984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eba334-a208-4d3f-baca-96446ff48bc0 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:08.759402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f810c-2df6-409c-839a-290520d46934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:08.760875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0f810c-2df6-409c-839a-290520d46934 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-ca0f810c-2df6-409c-839a-290520d46934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:08.777196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:08.778113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:08.778952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:08.778952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:08.778952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:08.778952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 237/469] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:08.779539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdf040-0fa2-4c48-91e8-59ea30ccdd70 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.779740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdf040-0fa2-4c48-91e8-59ea30ccdd70 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.780049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdf040-0fa2-4c48-91e8-59ea30ccdd70 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.781498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 238/470] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:08.785749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdf040-0fa2-4c48-91e8-59ea30ccdd70 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:08.808320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 233/471] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:46:08 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:08.811514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8ae06-05e1-4244-9015-ac65ae6f44d9 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.820981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8ae06-05e1-4244-9015-ac65ae6f44d9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.820981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8ae06-05e1-4244-9015-ac65ae6f44d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.836338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-3ed8ae06-05e1-4244-9015-ac65ae6f44d9 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:08.837553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3ed8ae06-05e1-4244-9015-ac65ae6f44d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:08.875491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:08.894803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:08.906411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdf040-0fa2-4c48-91e8-59ea30ccdd70 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:08.918421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 234/472] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:08.919667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8ae06-05e1-4244-9015-ac65ae6f44d9 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:08.944257 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:08.945216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:08.945636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:08.946054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:08.946886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:08.947324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:08.949784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:08.966197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 239/473] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:08.972643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:09.069265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8ae06-05e1-4244-9015-ac65ae6f44d9 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:09.104686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:09.105324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:09.105607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:09.105883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:09.116631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:09.118084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:09.134004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:09.135006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:09.135817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:09.135817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:09.135817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:09.135817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 235/474] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:09.141961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e964eb-c3d9-4b40-9948-929a1162d862 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:09.147344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:09.183782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:09.194377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:09.195055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:09.199303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:09.199303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:09.199303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:09.199303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:09.199599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:09.269841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464615be-d3d0-4478-81aa-0f2a7c055667 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:09.302265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 240/475] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:46:08 2022] POST /identity/v3/users => generated 248 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:46:09.302265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:09.319586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:09.322042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:09.322383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:09.322666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:09.324162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:09.324471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:09.325996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:09.355954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 236/476] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:09.364427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:09.495377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d00201-0bf7-40d1-9928-2d3653d9c7f9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:09.528259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:09.540874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f751c5e-391d-456f-9f30-bae6df91dc7c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:09.542855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:09.548059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:09.548059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:09.548059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:09.548059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:09.548059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:09.580958 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:09.581568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:09.581849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:09.582123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:09.582846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:09.584308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:09.589311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.sql.core [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 10:46:09.589311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 10:46:09.589311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [parameters: {'id': '6aba94b0f483470db0d361f06d0f833d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 10:46:09.589311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81529) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application conn.query(q) Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result.read() Aug 22 10:46:09.606058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return ref.to_dict() Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 10:46:09.607032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application next(self.gen) Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application yield resource Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application next(self.gen) Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise self.value Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application session.commit() Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 10:46:09.607975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.session.flush() Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application compat.raise_( Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 10:46:09.608952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 10:46:09.609877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application conn.query(q) Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result.read() Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 10:46:09.611903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [parameters: {'id': '6aba94b0f483470db0d361f06d0f833d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:09.612704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 22 10:46:09.613466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:09.614307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 241/477] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 22 10:46:09 2022] POST /identity/v3/roles => generated 144 bytes in 308 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:09.617424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:09.670321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 237/478] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:46:09 2022] POST /identity/v3/users => generated 249 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:09.675110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:09.737683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502658c-235f-47ef-820e-2b3639ac79c8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:09.778558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998bca6-1707-4ea3-8d98-6594cf750956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:09.779603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998bca6-1707-4ea3-8d98-6594cf750956 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:09.780077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998bca6-1707-4ea3-8d98-6594cf750956 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:09.780566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998bca6-1707-4ea3-8d98-6594cf750956 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:09.795737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998bca6-1707-4ea3-8d98-6594cf750956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-6998bca6-1707-4ea3-8d98-6594cf750956 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:09.801504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:09.802480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 22 10:46:09.803255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:09.807661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 242/479] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:09.811379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:09.967265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41d51a-8daf-4e9c-924b-ffe75943d316 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:10.007659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c2bf7-4ad9-4e94-85af-b7eb8022f4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:10.008268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c2bf7-4ad9-4e94-85af-b7eb8022f4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:10.008555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c2bf7-4ad9-4e94-85af-b7eb8022f4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:10.008826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c2bf7-4ad9-4e94-85af-b7eb8022f4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:10.009491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c2bf7-4ad9-4e94-85af-b7eb8022f4b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:10.009792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c2bf7-4ad9-4e94-85af-b7eb8022f4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:10.011193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5c2bf7-4ad9-4e94-85af-b7eb8022f4b5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:10.019648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 238/480] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:10.985322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39021a-ee3a-4afd-9d01-70f8cc79a2cf None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:10.986890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39021a-ee3a-4afd-9d01-70f8cc79a2cf None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:10.987528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39021a-ee3a-4afd-9d01-70f8cc79a2cf None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:10.989595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 243/481] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:10.995023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabaff-d28d-4f61-9a55-94eff94321bd None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:10.995474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabaff-d28d-4f61-9a55-94eff94321bd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:10.995929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cabaff-d28d-4f61-9a55-94eff94321bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:11.023227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-28cabaff-d28d-4f61-9a55-94eff94321bd None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:11.024451 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-28cabaff-d28d-4f61-9a55-94eff94321bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:11.027621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:11.111885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 239/482] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:11.421280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755252f2-53dd-46a3-9e0a-7b91d205ce8e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:11.421918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755252f2-53dd-46a3-9e0a-7b91d205ce8e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:11.422433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755252f2-53dd-46a3-9e0a-7b91d205ce8e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:11.424350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 244/483] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:11.429425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00f8dd-7c8c-4ad4-b964-b963fcc891ad None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:11.429715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00f8dd-7c8c-4ad4-b964-b963fcc891ad None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:11.429990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00f8dd-7c8c-4ad4-b964-b963fcc891ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:11.455282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7a00f8dd-7c8c-4ad4-b964-b963fcc891ad None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:11.456499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7a00f8dd-7c8c-4ad4-b964-b963fcc891ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:11.543562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 240/484] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:11.556976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755252f2-53dd-46a3-9e0a-7b91d205ce8e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:11.659525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6947244-62af-4e63-876e-485c23558400 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:11.659836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6947244-62af-4e63-876e-485c23558400 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:11.660087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6947244-62af-4e63-876e-485c23558400 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:11.661686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 241/485] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:11.666761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190618-663c-4920-97fe-848131399b46 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:11.667053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190618-663c-4920-97fe-848131399b46 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:11.667327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190618-663c-4920-97fe-848131399b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:11.685563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b1190618-663c-4920-97fe-848131399b46 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:11.686817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b1190618-663c-4920-97fe-848131399b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:11.723389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755252f2-53dd-46a3-9e0a-7b91d205ce8e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:11.734297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:11.763570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:11.764204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:11.764481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:11.764777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:11.777936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:11.784088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:11.784965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:11.785719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 245/486] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:11.792840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:11.806598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:11.810367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 242/487] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:11.932801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f145706-ca5a-4575-89fa-84e65cc5ead2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:11.939413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1190618-663c-4920-97fe-848131399b46 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:11.982978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:11.983593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:11.983879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:11.984156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:11.984828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:11.985133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:11.987244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.000498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 246/488] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:12.002029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.050952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.069724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1190618-663c-4920-97fe-848131399b46 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.112040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.112662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.112939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.113211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.124237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:12.129605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:12.130525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:12.131275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 243/489] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:12.131902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.146076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a8f10-c4c2-468f-8d6c-7a5930248d0f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.190468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.191042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.191321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.191593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.200257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.205108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.206835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:12.222633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:12.223758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:12.224795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:12.224795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:12.224795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:12.226547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 247/490] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:12.227509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.257729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4d1bd-ea83-4f41-8c25-f0980447e0db None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.295103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.295963 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.296449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.296726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.308648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:12.313330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:12.314224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:12.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:12.319872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 244/491] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:12.320561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.325592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.343663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe62774-30fc-4a03-8b4a-9c89121c523b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.381026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e3dbc-4604-49d8-b8c9-f007102d2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.381633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e3dbc-4604-49d8-b8c9-f007102d2063 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.381936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e3dbc-4604-49d8-b8c9-f007102d2063 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.382179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e3dbc-4604-49d8-b8c9-f007102d2063 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.382869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e3dbc-4604-49d8-b8c9-f007102d2063 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:12.383164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e3dbc-4604-49d8-b8c9-f007102d2063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.384532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e3dbc-4604-49d8-b8c9-f007102d2063 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.397237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 248/492] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:12.397237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f69f7-9db2-45cd-88aa-8e4aa8a7781b None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.397237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f69f7-9db2-45cd-88aa-8e4aa8a7781b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.397237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f69f7-9db2-45cd-88aa-8e4aa8a7781b None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.397237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 249/493] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:12.397237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f69f7-9db2-45cd-88aa-8e4aa8a7781b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.435074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7ad40-eb84-4ca7-ae36-27ae38ea7dc6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.458379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.474281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.475517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.475517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.475517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.476043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:12.476343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.477721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.497226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 245/494] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:12.497956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.515286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f69f7-9db2-45cd-88aa-8e4aa8a7781b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.552515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af4932-f148-4cd5-bb3d-8bb08c0327a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.553297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4932-f148-4cd5-bb3d-8bb08c0327a5 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.553676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4932-f148-4cd5-bb3d-8bb08c0327a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.554149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af4932-f148-4cd5-bb3d-8bb08c0327a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.554985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4932-f148-4cd5-bb3d-8bb08c0327a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:12.555364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4932-f148-4cd5-bb3d-8bb08c0327a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.556804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af4932-f148-4cd5-bb3d-8bb08c0327a5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.566213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 250/495] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:12.567770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3b1d2-f656-4778-a966-1aa8564f18df None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.568108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3b1d2-f656-4778-a966-1aa8564f18df None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.568598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3b1d2-f656-4778-a966-1aa8564f18df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.582892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.589402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-fab3b1d2-f656-4778-a966-1aa8564f18df None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:12.591325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fab3b1d2-f656-4778-a966-1aa8564f18df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:12.607180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b95e2-7c22-43b9-9f19-b947e78be461 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.644212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.644964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.645301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.645581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.656433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.658347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.667671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 251/496] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:12.668496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3b1d2-f656-4778-a966-1aa8564f18df None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:12.672130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:12.673621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:12.674453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:12.674453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:12.674453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:12.674453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 246/497] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:12.675060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.706488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.775936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88875cb-ab0e-4bf3-a9f6-f316bf7ebb33 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.785264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3b1d2-f656-4778-a966-1aa8564f18df None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.810994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a24a09-0bb6-4889-9ed0-ca741ad880c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.811703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a24a09-0bb6-4889-9ed0-ca741ad880c6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.812053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a24a09-0bb6-4889-9ed0-ca741ad880c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.812390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a24a09-0bb6-4889-9ed0-ca741ad880c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.822618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.823263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a24a09-0bb6-4889-9ed0-ca741ad880c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.823364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.823716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.824059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a24a09-0bb6-4889-9ed0-ca741ad880c6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.827779 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.833579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.834991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e8a24a09-0bb6-4889-9ed0-ca741ad880c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:12.839712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:12.840757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:12.841773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:12.841773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:12.841773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:12.845671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 247/498] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:12.846809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35110001-e526-42d9-a7a6-dde56863d413 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.847095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35110001-e526-42d9-a7a6-dde56863d413 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.847368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35110001-e526-42d9-a7a6-dde56863d413 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.848748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 248/499] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:12.849389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:12.850470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:12.851636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:12.852242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 252/500] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:12.853454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35110001-e526-42d9-a7a6-dde56863d413 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.858303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:12.944557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:12.961967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35110001-e526-42d9-a7a6-dde56863d413 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.964289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a22e2-534b-49a0-811a-9b798b47d0dd None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:12.997499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.998112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ccaaa-77d3-4255-84d9-b96e177c4a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:12.998251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.998342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ccaaa-77d3-4255-84d9-b96e177c4a62 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:12.998423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.998502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ccaaa-77d3-4255-84d9-b96e177c4a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:12.998657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.998774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ccaaa-77d3-4255-84d9-b96e177c4a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:12.999292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:12.999380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ccaaa-77d3-4255-84d9-b96e177c4a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:12.999587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ccaaa-77d3-4255-84d9-b96e177c4a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:12.999673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.001074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.001182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ccaaa-77d3-4255-84d9-b96e177c4a62 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.013479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 249/501] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:13.014791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c991-0fa0-49ef-bdd2-69860338e54e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.015072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c991-0fa0-49ef-bdd2-69860338e54e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.015171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 253/502] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:13.015660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2c991-0fa0-49ef-bdd2-69860338e54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.015960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.034354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-09e2c991-0fa0-49ef-bdd2-69860338e54e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:13.035467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-09e2c991-0fa0-49ef-bdd2-69860338e54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:13.076523 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.100878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 250/503] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:13.101777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2c991-0fa0-49ef-bdd2-69860338e54e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.122026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7828b1-c156-42d2-a8b8-aa315c1be64d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.160873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.161841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.162425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.162896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.163813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:13.164328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.166060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.185926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.208726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 254/504] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:13.208726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.215024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2c991-0fa0-49ef-bdd2-69860338e54e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.251013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.252042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.252330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.252656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.263495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:13.268676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:13.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:13.270399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 251/505] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:13.271895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.317320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa590479-4d51-4614-8a38-d3aada045700 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.332815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.353312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.353939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.354216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.354513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.362092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.363594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.374521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef65fb7-4ea1-49a8-a240-6312feee580c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:13.375767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:13.376645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:13.377508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 255/506] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:13.378073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.414712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.415338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.415626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.415902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.417785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.423481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.424997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:13.437350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:13.438549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 252/507] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:13.439630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.480398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e73a2-596b-4815-993c-8c360fa4fdbc None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.514409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.515021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.515297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.515568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.540534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132fcaa-0390-4473-a8ae-1e622241838b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.573507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.574031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.574337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.574609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.575266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:13.575565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.576939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.584932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 253/508] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:13.585741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.604570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.629914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.631447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.643754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.687168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f966ec8-168b-4393-9e81-894f40af4bfb None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.721185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.721799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.722077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.722380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.728825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 256/509] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:13.729713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.732612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.734330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:13.749852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:13.751022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:13.751993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:13.751993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:13.751993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:13.751993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 254/510] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:13.752197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.807008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.830929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11790d87-3338-4e90-8a7d-75f3e77a82bf None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.851123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c92e52-bb55-4cc9-98d0-2ed8e11761f6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:13.860326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:13.874301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.875359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.876383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.876880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.878171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:13.878770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.881380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.887545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:13.888171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:13.888453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:13.888729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:13.889478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:13.889788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:13.891389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:13.902126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 257/511] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:13.902994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:13.933272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 255/512] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:13.934106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.004476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3807db-58ee-4e44-8185-b3b3ca3a9448 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.033530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.033686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b8015f-5b37-4fb9-962d-c987df55ef74 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.038160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.038813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.039091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.039361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.050031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.051505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:14.065964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:14.067053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:14.067888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:14.067888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.067888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:14.072039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 258/513] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:14.072874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.073005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.073635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.073910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.074182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.086050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.087624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:14.102531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:14.104013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:14.105210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:14.105210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.105210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:14.105210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 256/514] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:14.105210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.123209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.176494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354a1db-83a4-47c6-84ec-018cce8a42d0 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.202054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb92df-e477-43c5-9cdb-5e0f1880ad6b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.209324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.210128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.210442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.210715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.211355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:14.211656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.213020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.225243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 259/515] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:14.226069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.239817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.240420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.240700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.240975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.270454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.306976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.308455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.323252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc33fa3-6a57-49f2-97d6-de76756e6298 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.326945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.356564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.357099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.357372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.357663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.409882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 257/516] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:14.410570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.431103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.432663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.475185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.506907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda7e67-a992-4021-acd5-bb0c31b700d1 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.524960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.531262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 260/517] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:14.532408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.542493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.543264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.543547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.543829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.544643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:14.544957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.546717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.558109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 258/518] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:14.558923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.637653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ff212-88ae-420f-b628-db577c6b809f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.657722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab84b-e0fe-434d-83c0-cc4c38bd3b67 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.670924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.671530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.671805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.673349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:14.673651 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.675049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.686366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 261/519] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:14.687292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.690831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.695586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.696190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.696478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.696743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.697380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.698836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.753911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 259/520] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:46:14 2022] POST /identity/v3/users => generated 247 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:14.754982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.789812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc2e16c-63f7-46f3-b1f1-007b29d82d16 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.805125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.832953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6036465-53e4-424c-8108-1754190533ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.833623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6036465-53e4-424c-8108-1754190533ac None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.833907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6036465-53e4-424c-8108-1754190533ac None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.834181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6036465-53e4-424c-8108-1754190533ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.846843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6036465-53e4-424c-8108-1754190533ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.848342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6036465-53e4-424c-8108-1754190533ac None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-d6036465-53e4-424c-8108-1754190533ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:14.864297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:14.865279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:14.866112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:14.866112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.866112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:14.870521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 262/521] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:14.871390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6036465-53e4-424c-8108-1754190533ac None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.891807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e3772-bc38-4e49-8beb-ec5ea995d2c3 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:14.935767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:14.936410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:14.936683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:14.936951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:14.948674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:14.948904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:14.950453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:14.966586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:14.970844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:14.971783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:14.971783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:14.971783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:14.971783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 260/522] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:14.971783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:14.996530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6036465-53e4-424c-8108-1754190533ac None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.043023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.043650 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.043937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.044210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.052488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.053977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:15.065803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:15.066718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:15.067589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 263/523] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:15.068121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.074699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:15.090581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43c921-300c-4980-ba39-2fd933a99cf1 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.126533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.127270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.127748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.128396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.135647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.137211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:15.149203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:15.150274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:15.151312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:15.154511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 261/524] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:15.155525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.179145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124f4a9-bb6b-44ac-8c96-c38a16100176 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.197218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:15.217866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.218518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.218799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.219072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.220373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:15.220802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.222278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.235176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 264/525] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:15.236692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.279774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa349ec-77ce-4127-8248-70780bf31c47 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.331233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.332172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.332625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.333034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.334093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:15.334622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.336961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.357086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 262/526] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:15.358461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.368653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:15.389319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0885cd-b363-4aa2-a238-6eaa3a1410b4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.431782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.433165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.433474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.433758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.434505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:15.434920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.436300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.464056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 265/527] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:15.464996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.479878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bda99-eba3-48ed-8a02-2563b60b5dcc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.507283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:15.521402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.522112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.522427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.524058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.524782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:15.526177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.528668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.555889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 263/528] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:15.556750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.593485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f9efa-e19f-4dff-bba3-402905476d1d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.635735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:15.643908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.644551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.644828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.645099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.653660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.653862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:15.672948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:15.674058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:15.675656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:15.679097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 266/529] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:15.680070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2f6db-751f-4b2d-a62b-c36a1a263b9d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.680217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.722500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.723131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.723410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.723680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.730576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.732023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:15.744084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:15.744982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:15.746116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:15.750075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 264/530] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:15.750983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.804110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:15.812646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c81ea1-6f23-4287-bdf9-070cfcbea056 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.854519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.855418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.855765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.856113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.879408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b553a1-82d0-48c2-a2ac-b583e62ad108 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:15.909886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:15.920478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:15.921286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:15.921563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:15.922562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:15.922840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7f5fe23bc8294bc3861e0db9dad2045e) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:15.923149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.924878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:15.934307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 265/531] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/role_assignments?user.id=7f5fe23bc8294bc3861e0db9dad2045e&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:15.934307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:15.945991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:15.947524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.090057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97115ee4-525c-4739-b4e7-d4c5de530cb6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:16.131015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:16.142547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 267/532] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:16.143609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:16.149000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:16.150021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.150604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.151130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.152041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:16.152410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:16.154147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.169408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 266/533] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:16.175844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:16.260177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:16.303400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f486e26-20dc-4ded-8355-e41147cb9784 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:16.317988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7425386-5578-4c9a-93cd-22ef6baecc67 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:16.351707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:16.353018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.353403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.353905 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.354814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:16.355191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:16.356732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.358580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:16.359445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.360480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.361013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.372242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:16.373970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.378717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 268/534] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:16.379595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:16.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:16.392455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:16.393256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:16.393256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:16.393256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:16.395152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 267/535] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:16.396468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:16.451223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:16.506250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7fc83-0eca-4320-8da1-8eea4d304da0 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:16.524452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ef69e-db4d-40c4-9d15-a32b3cff6c9a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:16.526363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:16.560820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:16.561884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.562568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.563036 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.564043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7cdb62860b6243bb84c31d1caff4bc77) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:16.564600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:16.565720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:16.566660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.567161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.567660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.567972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.568805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=990660ce5a9f4213a5b42b3824d0b567) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:16.569258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:16.570848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.575884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 269/536] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:16 2022] GET /identity/v3/role_assignments?user.id=7cdb62860b6243bb84c31d1caff4bc77&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:16.576986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:16.583717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 268/537] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:16 2022] GET /identity/v3/role_assignments?user.id=990660ce5a9f4213a5b42b3824d0b567&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:16.702878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503012b5-56f0-47d8-b00d-228a49754461 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:16.725308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:16.744283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e2c3e-646b-4002-978b-b9ad59fd0daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:16.745156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e2c3e-646b-4002-978b-b9ad59fd0daf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.745847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e2c3e-646b-4002-978b-b9ad59fd0daf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.746289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e2c3e-646b-4002-978b-b9ad59fd0daf None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.747172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e2c3e-646b-4002-978b-b9ad59fd0daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:16.747616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e2c3e-646b-4002-978b-b9ad59fd0daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:16.749209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e2c3e-646b-4002-978b-b9ad59fd0daf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.763814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 270/538] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:16.769627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:16.819835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02db23-1369-42b2-b3c6-a4056e3c116c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.820503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02db23-1369-42b2-b3c6-a4056e3c116c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.820974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02db23-1369-42b2-b3c6-a4056e3c116c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.823939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 271/539] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:16.830179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507aac3-6bcf-4ea3-bde5-b8cc1a299056 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.830614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507aac3-6bcf-4ea3-bde5-b8cc1a299056 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.830894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507aac3-6bcf-4ea3-bde5-b8cc1a299056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.859533 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-1507aac3-6bcf-4ea3-bde5-b8cc1a299056 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:16.861103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1507aac3-6bcf-4ea3-bde5-b8cc1a299056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:16.907292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e8a19-732f-4f49-88fb-b930c75e895e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:16.950632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:16.951647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:16.952172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:16.952638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:16.965538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:16.969031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:16.992784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:16.993648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:16.994537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:16.997705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 269/540] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:17.003407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:17.015844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 272/541] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:17.033573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:17.111398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c413d55-5fa1-4581-b747-0dc8df53ea37 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:17.147737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:17.148346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:17.148623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:17.148892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:17.149589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:17.150100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:17.151351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:17.164605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 270/542] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:17.170575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507aac3-6bcf-4ea3-bde5-b8cc1a299056 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:17.310948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507aac3-6bcf-4ea3-bde5-b8cc1a299056 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:17.326420 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:17.362380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:17.363282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:17.363737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:17.364182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:17.365105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=edd00e00b48641c8bcaacaaa7c2dd149) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:17.365588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:17.367230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:17.376679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 273/543] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:17 2022] GET /identity/v3/role_assignments?user.id=edd00e00b48641c8bcaacaaa7c2dd149&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:17.437709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:17.549176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:17.566797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcd074-edcc-45b4-b111-2918da429a83 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:17.608898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:17.609535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:17.609873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:17.610282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:17.623076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:17.628310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:17.629159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:17.629950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:17.630560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 271/544] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:17.637010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:17.753662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06af736-42d2-4f4a-90da-57d60f5156fb None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:17.788139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:17.788749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:17.789078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:17.789353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:17.790018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:17.790348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:17.791719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:17.800997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 274/545] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:17.806483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:17.970196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106f3e5-5f98-4668-ae78-3965b6d781a8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:18.030165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7c2d4-c98e-4a2c-bb91-282804bcde0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:18.030856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7c2d4-c98e-4a2c-bb91-282804bcde0b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.031136 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7c2d4-c98e-4a2c-bb91-282804bcde0b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.031409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7c2d4-c98e-4a2c-bb91-282804bcde0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.052686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7c2d4-c98e-4a2c-bb91-282804bcde0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:18.054171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7c2d4-c98e-4a2c-bb91-282804bcde0b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:18.061654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-47d7c2d4-c98e-4a2c-bb91-282804bcde0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:18.070243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:18.071123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:18.072138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:18.072138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:18.072138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:18.072138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 272/546] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:18.077162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:18.113300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbe046-7c9e-499f-8389-cb8ed728f358 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.113558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbe046-7c9e-499f-8389-cb8ed728f358 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.113778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbe046-7c9e-499f-8389-cb8ed728f358 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.115408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 273/547] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:18.124483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b7966-f14f-4d2f-b3b0-8644de42c5e2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.124773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b7966-f14f-4d2f-b3b0-8644de42c5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.125049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b7966-f14f-4d2f-b3b0-8644de42c5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.127545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:18.159256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-0e9b7966-f14f-4d2f-b3b0-8644de42c5e2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:18.160531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0e9b7966-f14f-4d2f-b3b0-8644de42c5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:18.193966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f54fd1-60ac-4e13-a044-8b9d8efc27f7 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:18.230614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:18.231574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.231857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.232128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.232780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:18.233077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:18.234478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:18.247809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 275/548] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:18.253742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:18.356196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:18.360554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941ab8-4ff2-4653-864e-00f1f99c5c9b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:18.374436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 274/549] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:18.396445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:18.397208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.397609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.397980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.405207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:18.407431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:18.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:18.420774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:18.421664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 276/550] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:18.431326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b7966-f14f-4d2f-b3b0-8644de42c5e2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:18.605029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b7966-f14f-4d2f-b3b0-8644de42c5e2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:18.640724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:18.654495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:18.655462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.655918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.656366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.657459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:18.657941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:18.659577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:18.688582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 275/551] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 22 10:46:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:18.694510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:18.709001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:18.810148 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cb180-7ba7-44d5-942d-5ffde3f936a1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:18.810726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799c6d-b7ce-4d78-bd9d-ded45336b669 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:18.853081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:18.853081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.853081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.853081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.861103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf003b0-60f1-466a-8fe8-7d393c2f943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:18.861103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf003b0-60f1-466a-8fe8-7d393c2f943e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:18.861103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf003b0-60f1-466a-8fe8-7d393c2f943e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:18.861103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf003b0-60f1-466a-8fe8-7d393c2f943e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:18.861490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:18.865746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:18.866658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:18.868755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 276/552] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:18.878257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:18.904214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:18.923489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:18.978165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01390b-e912-4ffb-8532-bb83446b8c14 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:18.990837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf003b0-60f1-466a-8fe8-7d393c2f943e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:18.992305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf003b0-60f1-466a-8fe8-7d393c2f943e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.015682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.017022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.017433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.017756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.018547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:19.018946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:19.020519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.030026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 277/553] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:19.036478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.156262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 277/554] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:19.158104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c497a5e-9f8a-4668-becf-e9d1519984ba None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.158264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a34dc-bd81-4df3-b360-7d1496fdf69d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:19.158412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c497a5e-9f8a-4668-becf-e9d1519984ba None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.158691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c497a5e-9f8a-4668-becf-e9d1519984ba None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.164233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 278/555] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:19.165112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c497a5e-9f8a-4668-becf-e9d1519984ba None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.168016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:19.198810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6b0bc-7d05-4a7f-bd33-07c0706f6a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.199597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b0bc-7d05-4a7f-bd33-07c0706f6a1f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.200219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b0bc-7d05-4a7f-bd33-07c0706f6a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.201001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6b0bc-7d05-4a7f-bd33-07c0706f6a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.215288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6b0bc-7d05-4a7f-bd33-07c0706f6a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:19.217486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6b0bc-7d05-4a7f-bd33-07c0706f6a1f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.229760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ebc6b0bc-7d05-4a7f-bd33-07c0706f6a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:19.236238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:19.237362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:19.238206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:19.238206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:19.238206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:19.242338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 278/556] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:19.245286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517de6e7-2612-4d0c-a7b3-6d719dfe5d6c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.245286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517de6e7-2612-4d0c-a7b3-6d719dfe5d6c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.245286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517de6e7-2612-4d0c-a7b3-6d719dfe5d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.268125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-517de6e7-2612-4d0c-a7b3-6d719dfe5d6c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:19.269627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-517de6e7-2612-4d0c-a7b3-6d719dfe5d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:19.291252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c497a5e-9f8a-4668-becf-e9d1519984ba None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:19.328065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be57993-1c28-413e-9c0d-7bd4d00f5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.328732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be57993-1c28-413e-9c0d-7bd4d00f5d05 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.329013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be57993-1c28-413e-9c0d-7bd4d00f5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.329300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be57993-1c28-413e-9c0d-7bd4d00f5d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.340478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be57993-1c28-413e-9c0d-7bd4d00f5d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:19.341920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be57993-1c28-413e-9c0d-7bd4d00f5d05 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.350432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 279/557] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:19.350432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517de6e7-2612-4d0c-a7b3-6d719dfe5d6c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-1be57993-1c28-413e-9c0d-7bd4d00f5d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:19.365567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:19.367187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:19.368255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:19.368255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:19.368255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:19.368793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 279/558] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:19.370032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b523b-4b8a-41fc-9ce2-552d9273aa4e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.370789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b523b-4b8a-41fc-9ce2-552d9273aa4e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.371140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b523b-4b8a-41fc-9ce2-552d9273aa4e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.372810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 280/559] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:19.383708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682898f-6547-4d09-9dec-966613a642e4 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.383998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682898f-6547-4d09-9dec-966613a642e4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.384251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682898f-6547-4d09-9dec-966613a642e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.412114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2682898f-6547-4d09-9dec-966613a642e4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:19.413758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2682898f-6547-4d09-9dec-966613a642e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:19.449795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:19.459856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517de6e7-2612-4d0c-a7b3-6d719dfe5d6c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:19.494473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 281/560] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:19.495359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682898f-6547-4d09-9dec-966613a642e4 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.503543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.504227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.504507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.504780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.505437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:19.505737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:19.515243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.537935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 280/561] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:19.543427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.555040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:19.670362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682898f-6547-4d09-9dec-966613a642e4 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:19.690510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056bc31-4dbf-4e55-877a-66a02c8fa029 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:19.715064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.716090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.716869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.717332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.718109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:19.718800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:19.720900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.731316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.732186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.732614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.733034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.738307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 282/562] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:19.739371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.746329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:19.748088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:19.761482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:19.762476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:19.763424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:19.764442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 281/563] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:19.765483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.779825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:19.838786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:19.844428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4231d8-11b2-4a15-ae66-a56e4d4978ef None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:19.872488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01757-5c99-4e52-bfb5-5ec63614a82c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:19.880527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c19dc-9caa-466a-b891-fd85c8405011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.881162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c19dc-9caa-466a-b891-fd85c8405011 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.881438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c19dc-9caa-466a-b891-fd85c8405011 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.881709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c19dc-9caa-466a-b891-fd85c8405011 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.892261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c19dc-9caa-466a-b891-fd85c8405011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-673c19dc-9caa-466a-b891-fd85c8405011 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:19.897066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:19.897970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:19.899093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 283/564] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:19.899792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a1826-a0d6-4015-bb47-0424e62250f7 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.899792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a1826-a0d6-4015-bb47-0424e62250f7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.899912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a1826-a0d6-4015-bb47-0424e62250f7 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.901181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 284/565] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:19.901705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a1826-a0d6-4015-bb47-0424e62250f7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:19.910605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:19.911233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:19.911509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:19.911779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:19.918552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:19.919990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:19.931709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:19.932733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 282/566] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:19.933583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.008102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.020114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a1826-a0d6-4015-bb47-0424e62250f7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.039450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab394d1-7a8e-47d6-87df-6bd7705846cc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.054581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.055189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.055471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.055796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.056597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:20.056899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.058382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.073318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e84d6b-3e27-4263-bc85-e5af5fcde5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.073945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84d6b-3e27-4263-bc85-e5af5fcde5fb None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.074224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84d6b-3e27-4263-bc85-e5af5fcde5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.074537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e84d6b-3e27-4263-bc85-e5af5fcde5fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.075184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 285/567] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:20.075804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.090008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e84d6b-3e27-4263-bc85-e5af5fcde5fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:20.094734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-56e84d6b-3e27-4263-bc85-e5af5fcde5fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:20.095368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:20.096243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 283/568] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:20.099389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39665ba6-e605-4405-8717-7ac096a9b968 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.100144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39665ba6-e605-4405-8717-7ac096a9b968 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.100144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39665ba6-e605-4405-8717-7ac096a9b968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.116036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-39665ba6-e605-4405-8717-7ac096a9b968 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:20.117239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-39665ba6-e605-4405-8717-7ac096a9b968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:20.181154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d6def-faf1-4129-bf5d-3b788986edda None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.182122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 284/569] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:20.182919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39665ba6-e605-4405-8717-7ac096a9b968 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.216887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.217501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.217780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.218069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.218742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:20.219097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.220459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.230202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 286/570] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:20.231019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.249714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.293897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39665ba6-e605-4405-8717-7ac096a9b968 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.320234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.331999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.332615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.336061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.336061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.336061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:20.336061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.336061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.345249 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df72a02-214e-40e2-9b0b-1d8da73c4b7d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.347759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 285/571] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:20.348588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.393190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.394501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.394781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.395051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.407325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.408937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:20.423978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:20.425109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:20.426205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:20.426205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:20.426205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:20.429792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 287/572] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:20.430560 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.460764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81241ddf-2b5d-4a9c-b880-796cb0586f27 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.480872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.494318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.494983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.495270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.495831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.496463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:20.496761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.498117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.506849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 286/573] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:20.507642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.536222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ed98-2fde-4b21-8678-dd4c752435bd None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.562031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.570605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.571237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.571517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.571792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.581899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.583373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:20.598384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:20.599295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:20.600134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:20.600134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:20.600134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:20.600134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 288/574] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:20.600425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.611401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c42de-974c-4da9-8451-77fa09d7e096 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.645066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.645753 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.646113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.646418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.647147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=c5e1b8292ac84020b6ac6db471a6e0ec) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:20.647458 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.648898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.654130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 287/575] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/role_assignments?user.id=c5e1b8292ac84020b6ac6db471a6e0ec&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:20.654934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.707799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c92d86-c5d7-4f94-9693-243abd1e74ae None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.722450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.747779 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.748413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.748689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.748960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.749618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:20.749920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.751941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.765263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 289/576] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:20.771742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.785718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b74a0c-7a6e-4a42-a270-3ba1be4e838e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.822099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.824833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.825468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.825749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.826019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.837485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:20.842619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:20.843534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 288/577] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:20.845608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.895818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d65d4-7600-49d7-93e5-9202cf15cf8f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.934331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.934966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.935237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:20.935501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:20.946459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:20.947917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:20.959724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbeba-ce82-49d3-9518-546ff85e4f52 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:20.964861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:20.965793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:20.966704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:20.966704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:20.966704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:20.966704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 290/578] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:20.967340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:20.971742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:20.999043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:20.999566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:20.999864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.000112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.001300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:21.001594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.002973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.018998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 289/579] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:21.018998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.083754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:21.094676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b55531-5df5-41b0-94b3-9b4d39e56172 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.133464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1360aa-1392-45c2-b60d-e12d964d7ef7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.133823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.134449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.134731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.135147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.143746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.145662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:21.159522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:21.160424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 291/580] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:21.166520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.173662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.174308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.174593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.174867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.175980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:21.176288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.178023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.191711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 290/581] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:21.191711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.225134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:21.277347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795988fd-62a2-4061-bd7f-903a85740ad1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.302105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a18193-1be5-496f-ba65-d6ab4ccab348 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.314535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.314821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.315099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.315742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:21.316047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.317414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.330361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 292/582] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:21.331157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.345098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.345971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.346447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.346890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.347460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:21.354360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:21.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:21.377618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 291/583] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:21.378601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.454170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072a17-e3e6-4772-a60b-8d326c96ac78 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.470355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:21.483245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3afd0d-f4de-4ef3-8e7b-df33adf85d4c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.489345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.490008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.490630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.490630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.491275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:21.491581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.493013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.509719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 293/584] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:21.509719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.525411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.526069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.526375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.526647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.537869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.539267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:21.553269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:21.554203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:21.555134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:21.555134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:21.555134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:21.559026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 292/585] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:21.559862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.590116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:21.620745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa5ca7-fb6a-4e5e-be71-36c0de761d82 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.655777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.656603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.656888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.657161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.664759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.666198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.667167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c431e-73b3-4e74-84d0-c0d024bdc6af None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:21.677864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:21.678991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 294/586] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:21.679908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.713967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.715081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.715416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.715697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.716444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:21.716839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.718739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.731237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:21.740474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 293/587] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:21.741455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.803094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f707cd9-9a85-45c3-9f6a-a3b6fe4d18ac None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.836973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:21.842084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.842684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.842959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.843243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7f5fe23bc8294bc3861e0db9dad2045e/roles/401c0ae4c0ad4b79bb877cbf745a71f2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.845990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b83492-0388-47f6-b7a5-5ad7fb5df574 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:21.861182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=7f5fe23bc8294bc3861e0db9dad2045e, role_id=401c0ae4c0ad4b79bb877cbf745a71f2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.862825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.885365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:21.885972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:21.886275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:21.886576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:21.887222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:21.887526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:21.888877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:21.900803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 294/588] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:21.901589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.936859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 295/589] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:21 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7f5fe23bc8294bc3861e0db9dad2045e/roles/401c0ae4c0ad4b79bb877cbf745a71f2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:21.937879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:21.998679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:22.030996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeff55-f1c8-4066-9aea-71a075f6a648 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.079696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.080424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.081090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.081484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.082600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:22.083011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.084639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.087911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049955d3-ec8b-4058-ab11-0607b0fff36f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.106756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 295/590] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:22.107757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.134500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.135398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.135676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.135946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.147928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.149366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:22.177218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:22.178425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:22.179576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:22.179576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:22.179576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:22.181432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 296/591] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:22.185170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.228118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:22.241596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a142d5-f672-4285-9668-2b2cb134567f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.264354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:22.286617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.287249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.287524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.287794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.296863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.299069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:22.310805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:22.311752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 10:46:22.312767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:22.317311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 296/592] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:22.318026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.334351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d49cc-f127-4c4b-a21c-68011a630ac5 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.382481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.383426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.383887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.384929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.400612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.403494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:22.425613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:22.426755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:22.427606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:22.427606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:22.427606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:22.433039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 297/593] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:22.433039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.443014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4196278-435d-4535-a42d-fe222d4b9586 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.479676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.480514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.480802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.481068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.481770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:22.482064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.483858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.502494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 297/594] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:22.503478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.544979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:22.555399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:22.577155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd6551-6526-4c18-b514-a30367d353aa None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.612757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4b37-9229-455e-a5f0-14df54066b19 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.618717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.619559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.620135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.620745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.621477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:22.621782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.623361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.643514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 298/595] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:22.644341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.652471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b9d5b-1785-4ce2-8ce0-aeef781ef533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.653083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b9d5b-1785-4ce2-8ce0-aeef781ef533 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.653364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b9d5b-1785-4ce2-8ce0-aeef781ef533 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.653636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b9d5b-1785-4ce2-8ce0-aeef781ef533 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.654299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b9d5b-1785-4ce2-8ce0-aeef781ef533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:22.654625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b9d5b-1785-4ce2-8ce0-aeef781ef533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.655973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b9d5b-1785-4ce2-8ce0-aeef781ef533 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.667859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 298/596] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:22.669077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cd766-15f1-426e-be67-ce808196db47 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.669362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cd766-15f1-426e-be67-ce808196db47 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.669633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cd766-15f1-426e-be67-ce808196db47 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.670983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 299/597] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:22.671511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cd766-15f1-426e-be67-ce808196db47 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.759967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfe919-c12d-436e-b484-dcbe314eabe7 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.771577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:22.778944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cd766-15f1-426e-be67-ce808196db47 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.795461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.796179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.796567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.796961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.804798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.806257 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.815983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8fb6c-3020-4820-82b3-c391797cba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.816610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8fb6c-3020-4820-82b3-c391797cba4e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.816954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8fb6c-3020-4820-82b3-c391797cba4e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.817232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8fb6c-3020-4820-82b3-c391797cba4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:22.818321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:22.819156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 299/598] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:22.824980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.829157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:22.829270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8fb6c-3020-4820-82b3-c391797cba4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:22.830694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8fb6c-3020-4820-82b3-c391797cba4e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-50c8fb6c-3020-4820-82b3-c391797cba4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:22.846649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:22.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:22.848443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:22.848443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:22.848443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:22.848443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 300/599] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:22.849074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9544-2741-4543-9561-e358fdac01f1 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.849370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9544-2741-4543-9561-e358fdac01f1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.849643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9544-2741-4543-9561-e358fdac01f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.875961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b07b9544-2741-4543-9561-e358fdac01f1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:22.877103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b07b9544-2741-4543-9561-e358fdac01f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:22.936664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f04a33-c576-4885-b614-141ff788600c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:22.947721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 301/600] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:22.949438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b9544-2741-4543-9561-e358fdac01f1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:22.987636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:22.988699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:22.988985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:22.989263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:22.999671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.001857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:23.019622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:23.020545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:23.021395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 300/601] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:23.022369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.047517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:23.075468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b9544-2741-4543-9561-e358fdac01f1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.111535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:23.116058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.116955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.117454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.117919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.118887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:23.119446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.121422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.138671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 302/602] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:23.139774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.158117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a1130-c265-49fe-b66b-acb8aabbcc6f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.205419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.206351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.207720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.208761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.209489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:23.210517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.212145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.231040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 301/603] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:23.231040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.264748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af5382-5aae-4c01-b909-13689c3ac463 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.300114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.300722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.301003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.301337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.302030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:23.302376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.303797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.310861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:23.322195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 303/604] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:23.322195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.343128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1820ae-3b6a-4e95-b55c-719d6c68f6a5 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.370663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:23.383039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.383783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.384127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.384484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/990660ce5a9f4213a5b42b3824d0b567/roles/401c0ae4c0ad4b79bb877cbf745a71f2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.399639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=990660ce5a9f4213a5b42b3824d0b567, role_id=401c0ae4c0ad4b79bb877cbf745a71f2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.401373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.440628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3d49-47e7-443e-96d6-37e567c03967 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.474348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 302/605] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:23 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/990660ce5a9f4213a5b42b3824d0b567/roles/401c0ae4c0ad4b79bb877cbf745a71f2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:23.474526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.475085 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.475368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.475632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.475980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.494525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.496450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:23.509853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:23.511046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 304/606] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:23.512262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.586540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:23.622868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2ad2b-d8d7-4078-a675-1bbd61193676 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.635060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df1b61-f43d-4d3b-9d5e-bf4772a505ed None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.666839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.667753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.668036 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.668570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.680423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.681393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:23.681515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.681852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.682204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7cdb62860b6243bb84c31d1caff4bc77/roles/401c0ae4c0ad4b79bb877cbf745a71f2` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:23.687832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:23.688954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:23.690042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:23.692222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 303/607] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:23.692941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.702034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=7cdb62860b6243bb84c31d1caff4bc77, role_id=401c0ae4c0ad4b79bb877cbf745a71f2)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.704282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.737023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:23.800894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 305/608] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:23 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7cdb62860b6243bb84c31d1caff4bc77/roles/401c0ae4c0ad4b79bb877cbf745a71f2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:23.801825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.824361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d598e1-2dcf-44c5-ae38-373546a793cd None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:23.908149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1767f2-96a0-4c14-833c-32e10c5c5f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:23.908877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1767f2-96a0-4c14-833c-32e10c5c5f4f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.909272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1767f2-96a0-4c14-833c-32e10c5c5f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.909619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1767f2-96a0-4c14-833c-32e10c5c5f4f None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.910433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1767f2-96a0-4c14-833c-32e10c5c5f4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:23.911041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1767f2-96a0-4c14-833c-32e10c5c5f4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:23.914503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1767f2-96a0-4c14-833c-32e10c5c5f4f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:23.932220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 304/609] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:23.933539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eee7f3-a607-46c9-8022-761aff100c09 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:23.933897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eee7f3-a607-46c9-8022-761aff100c09 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:23.934256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eee7f3-a607-46c9-8022-761aff100c09 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:23.935853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 305/610] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:23.938577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eee7f3-a607-46c9-8022-761aff100c09 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:23.990509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6d0fb-a084-486d-85c1-12f9664f4356 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.003814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:24.039806 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d77954-eb00-4b94-9d46-0a1574142940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:24.040659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77954-eb00-4b94-9d46-0a1574142940 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.040990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77954-eb00-4b94-9d46-0a1574142940 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.045575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77954-eb00-4b94-9d46-0a1574142940 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.046337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d77954-eb00-4b94-9d46-0a1574142940 None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:24.046656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d77954-eb00-4b94-9d46-0a1574142940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:24.048878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d77954-eb00-4b94-9d46-0a1574142940 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:24.058008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 306/611] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:24.059133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92c1e-3b84-4014-b1d7-cda18077a7c2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.059418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92c1e-3b84-4014-b1d7-cda18077a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.059695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92c1e-3b84-4014-b1d7-cda18077a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.074697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eee7f3-a607-46c9-8022-761aff100c09 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.078535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:24.082001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-84e92c1e-3b84-4014-b1d7-cda18077a7c2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:24.083307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-84e92c1e-3b84-4014-b1d7-cda18077a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:24.116374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:24.117055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.117382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.117661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/edd00e00b48641c8bcaacaaa7c2dd149/roles/401c0ae4c0ad4b79bb877cbf745a71f2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.134628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=edd00e00b48641c8bcaacaaa7c2dd149, role_id=401c0ae4c0ad4b79bb877cbf745a71f2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:24.136144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:24.159300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 307/612] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:24.160162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e92c1e-3b84-4014-b1d7-cda18077a7c2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:24.236153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 306/613] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:23 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/edd00e00b48641c8bcaacaaa7c2dd149/roles/401c0ae4c0ad4b79bb877cbf745a71f2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:24.275338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:24.279272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e92c1e-3b84-4014-b1d7-cda18077a7c2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.326822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:24.327481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.327817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.328136 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.339788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:24.341237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:24.358755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:24.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:24.361052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:24.361052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:24.361052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:24.367713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 308/614] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:24.379965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:24.482755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:24.533488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490961f-991c-4dc6-a06c-8eeee318ba8d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.571889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:24.572922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.572922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.573207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.573897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:24.574205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:24.575865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:24.594288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 307/615] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:24.596260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:24.600741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:24.608428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7712c74-968b-4d4a-852c-4a5056808b75 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.674922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:24.675565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.675845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.676119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.689809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:24.694998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:24.695917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:24.696715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 309/616] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:24.707223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:24.714182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f4169-b87a-4300-b5c5-f028a30c1177 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.733321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:24.758728 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:24.759817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.760334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.760900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.772629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:24.774656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:24.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:24.789394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:24.790324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:24.793043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 308/617] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:24.798747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:24.880987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:24.926583 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a961226-e4b7-473e-81d2-80452e4ceeb9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.939283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b49cbc-acad-4233-93e7-3d2a1e9b61ca None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:24.964441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:24.965430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:24.965934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:24.966500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:24.967853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:24.968363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:24.970817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:24.992348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 309/618] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 22 10:46:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:25.002162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.026019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.026662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.026945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.027218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.029042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:25.029347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.031775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.048588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 310/619] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:25.054041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.159806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46e607-7879-4cf8-accd-753f05928733 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:25.201668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:25.223077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.223715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.223990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.224260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.238784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:25.243887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.245653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:25.268751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:25.269872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:25.270791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:25.270791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:25.270791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:25.270791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 310/620] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:25.277275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.284654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b19f6b-b70e-4f3c-9ec4-239156e6a58a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:25.326270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.326849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.327131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.328675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.348488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.354263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:25.377319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:25.378380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:25.379264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:25.379264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:25.379264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:25.379264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 311/621] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:25.386839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.492349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08188d2e-65d9-4cf6-9f33-d6a3ecac7fe6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:25.509032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027ccfc-8ab9-4ae0-b213-e5093b0745e9 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:25.540489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.541106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.541386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.541662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.542342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:25.542648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.544029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.563137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:25.564051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 311/622] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:25.571025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.573413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.573413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.573413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.573413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.574120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:25.574550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.576146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.589868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 312/623] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:25.595411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.653167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:25.704027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b0a50-fa00-49d6-98f2-e29fb15bfb23 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:25.743454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff770c4e-4c00-4d07-9ab1-95525baf16cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.744380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff770c4e-4c00-4d07-9ab1-95525baf16cb None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.744789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff770c4e-4c00-4d07-9ab1-95525baf16cb None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.745163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff770c4e-4c00-4d07-9ab1-95525baf16cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.753831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff770c4e-4c00-4d07-9ab1-95525baf16cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.756127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff770c4e-4c00-4d07-9ab1-95525baf16cb None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ff770c4e-4c00-4d07-9ab1-95525baf16cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:25.769091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:25.770252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:25.771202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 313/624] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:25.771798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520146b-204c-4e29-a2be-6cbdb0b82d37 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.774280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520146b-204c-4e29-a2be-6cbdb0b82d37 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.774280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520146b-204c-4e29-a2be-6cbdb0b82d37 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.774280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 314/625] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:25.778979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520146b-204c-4e29-a2be-6cbdb0b82d37 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.813643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b04f9d-0542-461e-bcce-f8b0eaec9405 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:25.832093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:25.849085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea325b2a-ead9-4dbe-810d-4cf55fbc539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.849305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea325b2a-ead9-4dbe-810d-4cf55fbc539d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.849562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea325b2a-ead9-4dbe-810d-4cf55fbc539d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.849838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea325b2a-ead9-4dbe-810d-4cf55fbc539d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.860499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea325b2a-ead9-4dbe-810d-4cf55fbc539d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.864198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea325b2a-ead9-4dbe-810d-4cf55fbc539d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-ea325b2a-ead9-4dbe-810d-4cf55fbc539d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:25.878927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:25.880993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 312/626] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952a45f-2554-4e1d-807a-0c9e1998674a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.883175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952a45f-2554-4e1d-807a-0c9e1998674a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.884594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952a45f-2554-4e1d-807a-0c9e1998674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.893532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520146b-204c-4e29-a2be-6cbdb0b82d37 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:25.901019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9952a45f-2554-4e1d-807a-0c9e1998674a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:25.902219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9952a45f-2554-4e1d-807a-0c9e1998674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:25.930693 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:25.931306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:25.931581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:25.931853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:25.932669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:25.932973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:25.934589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:25.954914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 315/627] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:25.955944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.972599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 313/628] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:25.973458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952a45f-2554-4e1d-807a-0c9e1998674a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:25.998730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:26.075938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952a45f-2554-4e1d-807a-0c9e1998674a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.090475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54cdfd-b8ae-4061-89b0-20503d6d0cbe None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.092790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:26.113354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.113996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.114361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.114705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.129086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62285f3-1e9d-493f-a587-a4338a625597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.130046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62285f3-1e9d-493f-a587-a4338a625597 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.130570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62285f3-1e9d-493f-a587-a4338a625597 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.131073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62285f3-1e9d-493f-a587-a4338a625597 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.131953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62285f3-1e9d-493f-a587-a4338a625597 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:26.132528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62285f3-1e9d-493f-a587-a4338a625597 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:26.134205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62285f3-1e9d-493f-a587-a4338a625597 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:26.147607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 316/629] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:26.149251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1a11-29ec-45fe-b767-2ad8ca1dbd87 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.149751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1a11-29ec-45fe-b767-2ad8ca1dbd87 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.150261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1a11-29ec-45fe-b767-2ad8ca1dbd87 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.152100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 317/630] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:26.161382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830c3d6-1901-4315-ad50-f591a72127a6 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.161382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830c3d6-1901-4315-ad50-f591a72127a6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.161382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830c3d6-1901-4315-ad50-f591a72127a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.177032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6830c3d6-1901-4315-ad50-f591a72127a6 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:26.178317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6830c3d6-1901-4315-ad50-f591a72127a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:26.187554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:26.189028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:26.214759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:26.254579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 318/631] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:26.255819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830c3d6-1901-4315-ad50-f591a72127a6 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.293408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 314/632] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:26.299974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.387068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:26.394623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830c3d6-1901-4315-ad50-f591a72127a6 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.404588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2515af-68cc-4f6b-8ea4-1bd1951a0f5f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.430368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:26.434760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.435473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.435754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.436025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/c5e1b8292ac84020b6ac6db471a6e0ec/roles/401c0ae4c0ad4b79bb877cbf745a71f2` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.441955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.442626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.444263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.444263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.451580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=c5e1b8292ac84020b6ac6db471a6e0ec, role_id=401c0ae4c0ad4b79bb877cbf745a71f2)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:26.453055 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:26.454794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:26.456418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:26.472282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:26.473316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:26.474391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:26.474391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:26.474391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:26.474391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 315/633] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:26.474628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.530420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 319/634] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:26 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/c5e1b8292ac84020b6ac6db471a6e0ec/roles/401c0ae4c0ad4b79bb877cbf745a71f2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:26.532015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.606139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f7f45-7114-470a-9c6f-8d3103f72e15 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.646953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.647518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.647804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.648068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.661178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:26.666652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:26.667599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:26.668430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 316/635] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:26.669102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.696223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ea74c-2064-4686-a7bd-0cdf10cd6c90 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.722124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:26.733521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee190c-6c5c-42c6-bb61-c377fc5bede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.734750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee190c-6c5c-42c6-bb61-c377fc5bede2 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.735028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee190c-6c5c-42c6-bb61-c377fc5bede2 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.742656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee190c-6c5c-42c6-bb61-c377fc5bede2 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.742656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee190c-6c5c-42c6-bb61-c377fc5bede2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:26.742656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee190c-6c5c-42c6-bb61-c377fc5bede2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:26.742656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee190c-6c5c-42c6-bb61-c377fc5bede2 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:26.749374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 320/636] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:26.750666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389b18-db1b-48b1-89e2-e1e514eb9058 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.750949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389b18-db1b-48b1-89e2-e1e514eb9058 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.751220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f389b18-db1b-48b1-89e2-e1e514eb9058 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.752540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 321/637] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:26.753145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f389b18-db1b-48b1-89e2-e1e514eb9058 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.757506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:26.796104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac56f92-6234-4326-9013-0e60469780f3 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.832765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af5ee5-e9b4-4bfe-9b58-09601b5fec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.833547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af5ee5-e9b4-4bfe-9b58-09601b5fec47 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.833837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af5ee5-e9b4-4bfe-9b58-09601b5fec47 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.834928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af5ee5-e9b4-4bfe-9b58-09601b5fec47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.847323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af5ee5-e9b4-4bfe-9b58-09601b5fec47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-f3af5ee5-e9b4-4bfe-9b58-09601b5fec47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:26.855986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:26.861912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 317/638] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:26.863444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588fb0-c4a6-42c3-8f1b-ca61255d3d7c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.864192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588fb0-c4a6-42c3-8f1b-ca61255d3d7c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.864192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588fb0-c4a6-42c3-8f1b-ca61255d3d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.873394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f389b18-db1b-48b1-89e2-e1e514eb9058 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:26.882488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-02588fb0-c4a6-42c3-8f1b-ca61255d3d7c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:26.883688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-02588fb0-c4a6-42c3-8f1b-ca61255d3d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:26.911619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:26.912180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:26.912459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:26.912731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:26.913516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:26.913747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:26.915462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:26.923780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 322/639] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:26.924579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.963173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 318/640] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:26.971669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02588fb0-c4a6-42c3-8f1b-ca61255d3d7c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:26.996729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.009951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.036430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bb11-4798-4e59-b5b3-42ca1238adcf None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.075484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.076357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.076765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.077137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.084634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.086190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:27.098877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:27.099947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:27.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:27.103267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 323/641] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:27.103767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02588fb0-c4a6-42c3-8f1b-ca61255d3d7c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.103886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.155250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.155873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.156730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.157054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.157843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:27.158144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.160137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.174395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 319/642] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:27.175379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.233252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c8914-cd46-4a76-ba2f-fa97ed5ea920 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.266691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.277604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.278535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.278822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.279092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.285504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.291560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.293012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.293762 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129430c3-392b-442f-9d6b-5d2dde2c0436 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:27.307613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:27.308669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:27.309544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:27.309544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:27.309544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:27.311888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 324/643] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:27.313022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.334834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.335940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.336433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.337081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.338305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:27.338873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.340290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.357439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 320/644] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:27.357439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.420511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4203fd-6141-46ec-90a9-9b85a9cde89a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.462480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.463028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36c4d-5099-4422-b38f-0852811e2d11 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.463161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.463384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.463664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.476206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.477720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:27.491825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:27.492764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:27.494550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:27.494550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:27.494550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:27.494550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 325/645] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:27.494550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.507052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.507970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.508242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.508517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.519656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:27.524874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:27.525774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 321/646] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:27.527121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.533386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.542074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.615213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fde20-d4f2-4ce5-b22a-a5970f2a1979 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.635183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c833d-b962-4555-9518-9a74707e4b6b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.651826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.652441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.652725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.653001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.653651 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:27.653955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.655366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.668724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 326/647] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:27.669431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.679393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.680358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.680358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.680628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.681306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7f5fe23bc8294bc3861e0db9dad2045e) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:27.681616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.683120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.690659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 322/648] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/role_assignments?user.id=7f5fe23bc8294bc3861e0db9dad2045e&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:27.691999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.776466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.790709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:27.792433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddebb5-358e-4191-bff1-1415cc569782 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.836027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.836947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.837372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.837680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5775a-0031-42ba-a16d-85a7ca8d4c1e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:27.837974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.838826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:27.839267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.841232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.858224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 327/649] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:27.859782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.884609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:27.885926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:27.886456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:27.887217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:27.887899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:27.888207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:27.890054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:27.901509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 323/650] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:27.902386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:27.986745 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfb88d-ce20-4e4f-a968-382fc82b076f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.020885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bafb34-d4ae-4e34-ae8c-1b88d2d3227d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.024925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.025657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.026021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.026376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.036929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.039110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:28.055154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:28.056120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 328/651] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:28.057060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.068342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.069358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.069883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.070408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.072770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.080473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.082152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.084577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:28.099962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:28.100972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:46:28.101925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:28.103609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 324/652] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:28.106266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.194973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9c365-f170-4905-84ad-f20221caf17b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.222081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fae37d-02cc-40a5-a30a-00571632beeb None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.236530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.237479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.237479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.237768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.249190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.250805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.259396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.260285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.260704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.261140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.262009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:28.262494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.264061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:28.266535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:28.267610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:28.268593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:28.268593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:28.268593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:28.272579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 329/653] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:28.273453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.287528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 325/654] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:28.288626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.344150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.378643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.387497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5368d-4487-4735-96e2-97068e258343 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.396261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637b4be-03b5-4783-bbe2-64efa82346cd None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.424533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.425307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.425686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.426103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.426951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:28.427358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.428867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.435807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.436602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.437047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.438128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.438995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:28.439440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.440935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.453283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 326/656] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:28.453649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 330/656] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:28.454725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.454976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.565824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c8d7a-582f-4126-b227-ceefaf82e022 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.568225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7ab67-9d03-424b-8062-aa2b474f1c6c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.582194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.586826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.601967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05515ca9-4082-4c6f-89d5-2e6f09276543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.602612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05515ca9-4082-4c6f-89d5-2e6f09276543 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.602890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05515ca9-4082-4c6f-89d5-2e6f09276543 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.603163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05515ca9-4082-4c6f-89d5-2e6f09276543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.604957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.605564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.605840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.606109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.677230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.677847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05515ca9-4082-4c6f-89d5-2e6f09276543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.678792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.679093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05515ca9-4082-4c6f-89d5-2e6f09276543 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.785900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 327/657] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:28.785900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.788196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 331/658] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:28.789655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347277f-dfc4-4187-841f-7315704f589b None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.789996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347277f-dfc4-4187-841f-7315704f589b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.790306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347277f-dfc4-4187-841f-7315704f589b None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.794375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 332/659] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:28.794375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347277f-dfc4-4187-841f-7315704f589b None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.800721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.819671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:28.894733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390f59-1b2f-4f28-924a-bb50c9dacb46 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.915581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347277f-dfc4-4187-841f-7315704f589b None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:28.929652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.930561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.930835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.931106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.939151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.940595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.950756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d2b4e-a3a0-45a6-ae1c-27fbb0684ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:28.951602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d2b4e-a3a0-45a6-ae1c-27fbb0684ecc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.952053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d2b4e-a3a0-45a6-ae1c-27fbb0684ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:28.952426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:28.953946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 328/660] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:28.955094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:28.956161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d2b4e-a3a0-45a6-ae1c-27fbb0684ecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:28.969313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d2b4e-a3a0-45a6-ae1c-27fbb0684ecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:28.970797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d2b4e-a3a0-45a6-ae1c-27fbb0684ecc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-c08d2b4e-a3a0-45a6-ae1c-27fbb0684ecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:28.987221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:28.988241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:28.989108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:28.989108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:28.989108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:28.989108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 333/661] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:28.989839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c3971-e182-4d6a-9b46-15ef2d317936 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:28.990176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c3971-e182-4d6a-9b46-15ef2d317936 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:28.990468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c3971-e182-4d6a-9b46-15ef2d317936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.010366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-371c3971-e182-4d6a-9b46-15ef2d317936 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:29.011566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-371c3971-e182-4d6a-9b46-15ef2d317936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:29.039976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.071776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4f94e-44a7-495d-8c8a-a32c70a42bd9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.081786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 334/662] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:29.082875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c3971-e182-4d6a-9b46-15ef2d317936 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.085333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.121100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.122109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.122606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.123090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.139309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.141358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:29.160645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:29.161670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:29.162584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:29.162584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:29.162584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:29.168260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 329/663] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:29.168260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.204920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c3971-e182-4d6a-9b46-15ef2d317936 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.251008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.252217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.253286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.253781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.255082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:29.255589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.258327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.285375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 335/664] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:29.286813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.291176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61d0bd-52bc-4796-b971-90b22c64f60a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.316384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.334454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.334454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.334454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.334454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.334884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:29.334884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.334884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.350277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 330/665] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:29.351325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.364518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.408213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee5da5-42ac-40ee-8823-b452571b2fca None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.442077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.442786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.442980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.443251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.443899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:29.444201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.445572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.456749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 336/666] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:29.457574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.469787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5eb84-7413-4cab-b51e-07cd834519ef None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.507448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.508580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.509164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.509635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.559834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46416b1-4bc6-48d4-bb86-a93e079d80fd None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.568986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.581711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.586120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.587310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.596320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.596949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.597226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.597500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.604493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.605926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:29.617000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:29.618674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 337/667] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:29.619527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.703299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 331/668] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:29.704209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.734979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5aaa4-c9a6-4c86-b4c5-86187da31fa8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.770613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.771297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.771571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.771845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.782304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:29.787556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:29.788476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:29.789291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 338/669] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:29.789981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.794737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.813194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b38a40-36a6-48d2-a72e-32c24884ab32 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.835598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:29.848376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.848956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.849235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.849508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.856839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.858318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:29.870623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:29.871673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 332/670] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:29.872884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.901363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a9bd8-8609-4e09-8b2b-8a600ebc5560 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:29.934562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:29.935163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:29.935442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:29.935718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:29.936405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:29.936705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:29.938872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:29.950322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 339/671] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:29.951106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:29.971069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a358e-9e01-4886-843b-30950e6dd66e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.004242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146167fb-500c-4bfa-976b-f861e7a52187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.004869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146167fb-500c-4bfa-976b-f861e7a52187 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.005152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146167fb-500c-4bfa-976b-f861e7a52187 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.005421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146167fb-500c-4bfa-976b-f861e7a52187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.016260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146167fb-500c-4bfa-976b-f861e7a52187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.017770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146167fb-500c-4bfa-976b-f861e7a52187 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.020350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-146167fb-500c-4bfa-976b-f861e7a52187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:30.033543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:30.034587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:30.035471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:30.035471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:30.035471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:30.035471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 333/672] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:30.036162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa106b-df79-4f26-b820-e97587b620bf None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.036443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa106b-df79-4f26-b820-e97587b620bf None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.036712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa106b-df79-4f26-b820-e97587b620bf None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.037914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 334/673] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:30.038683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa106b-df79-4f26-b820-e97587b620bf None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.055765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.059648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c6fc8-5931-4be1-91c9-bbf0ff23c855 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.095659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.096268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.096548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.096824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.097489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:30.097784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.099251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.106997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 340/674] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:30.107853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.145901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa106b-df79-4f26-b820-e97587b620bf None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.179073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.179680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.179955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.180233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.180906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:30.181205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.182673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.194814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 335/675] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:30.195593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.210423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275d051-7e2f-48f6-8474-ad02b6325881 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.243091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64bcf-8f2e-4b6a-b88a-82e6d7f05e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.243684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64bcf-8f2e-4b6a-b88a-82e6d7f05e5b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.243960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64bcf-8f2e-4b6a-b88a-82e6d7f05e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.244232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64bcf-8f2e-4b6a-b88a-82e6d7f05e5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.244940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a64bcf-8f2e-4b6a-b88a-82e6d7f05e5b None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=990660ce5a9f4213a5b42b3824d0b567) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:30.245502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a64bcf-8f2e-4b6a-b88a-82e6d7f05e5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.246940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a64bcf-8f2e-4b6a-b88a-82e6d7f05e5b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.253138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 341/676] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/role_assignments?user.id=990660ce5a9f4213a5b42b3824d0b567&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:30.253400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.254358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdb182-d016-421b-838c-52aadb06e568 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.254740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdb182-d016-421b-838c-52aadb06e568 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.255018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdb182-d016-421b-838c-52aadb06e568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.284941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-decdb182-d016-421b-838c-52aadb06e568 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:30.287109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-decdb182-d016-421b-838c-52aadb06e568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:30.308940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c854e84-a906-4ebb-89e9-6bb2112ebc1c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.317066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.349007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.350016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.350269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.350828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.351499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:30.351804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.353788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.368141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 336/677] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:30.369381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.387164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 342/678] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:30.388606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdb182-d016-421b-838c-52aadb06e568 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.495578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba11bb-a92c-4dbb-9985-3cfc446a4a99 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.515961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.528374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdb182-d016-421b-838c-52aadb06e568 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.537957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.548763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.548763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.548763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.551731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.553945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.567420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.568034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.568325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.568596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.569316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7cdb62860b6243bb84c31d1caff4bc77) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:30.569624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.571076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:30.575362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:30.576303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:30.577669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:30.577669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:30.577669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:30.577837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 343/679] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/role_assignments?user.id=7cdb62860b6243bb84c31d1caff4bc77&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:30.578008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.579845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 337/680] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:30.588017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.639758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.739927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8b209-754d-44bb-a233-e437f34f230a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.743825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebb5d8-78fe-4139-ac86-822b925f072d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.791000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.791640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.791921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.792195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.793301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.794397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.794397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.794590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.795106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:30.795406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.796876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.802749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.804215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:30.812971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 338/681] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:30.813821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:30.818164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:30.819039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:30.819898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 344/682] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:30.820879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:30.859821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.922801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:30.946899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724deb3-e5e7-4bd0-ab23-42427b55b6a1 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.956266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb11ef0-7ffd-4c54-bd55-5bfd2f97ff26 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:30.983118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:30.983753 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:30.984080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:30.984360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:30.991978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:30.993580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:31.003315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.004230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.004742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.005214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:31.006093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:31.007043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:31.007953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:31.012136 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 345/683] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:31.013031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.021547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:31.028047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:31.028978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:31.029786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:31.032271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 339/684] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:31.032868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.131867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b5d2a-e9c1-4e04-95e2-493a759fffb6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.160193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:31.179893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7639c8e-73c6-43bb-96c3-52c951a71e76 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.180241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.181101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.181723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.183054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.183971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:31.184499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:31.186152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:31.200003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:31.205710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 346/685] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:31.207477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.217334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.218031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.218567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.218908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.219778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:31.220147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:31.221705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:31.246808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 340/686] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:31.247901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.331902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee735b8-b1df-4851-8c52-cc8091980c39 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.368466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.369258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.369446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.369841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.370855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:31.371271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:31.372750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:31.377128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495b86c-e0d7-449e-9205-91592ede4e62 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.383215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 347/687] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:31.384262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.414627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.415238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.415516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.415789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.416521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=edd00e00b48641c8bcaacaaa7c2dd149) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:31.416871 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:31.418369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:31.425241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 341/688] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/role_assignments?user.id=edd00e00b48641c8bcaacaaa7c2dd149&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:31.426066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.456769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:31.467803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:31.520214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50a13b-ae4a-47f6-b3bd-fb39f9eaad54 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.561766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.564014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.564485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.565175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.646387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:31.648203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:31.685938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e7b2d-a78c-4669-b55c-9fabfdc2a04c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.729182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.729825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.730102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.730419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.739088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:31.741649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:31.743179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:31.760996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:31.762005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:31.762944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:31.762944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:31.762944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:31.762944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 342/689] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:31.768821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.799067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 348/690] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:31.805057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:31.855856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:31.914047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6f3c8-c1cd-47a9-9c8e-78450d7c29fa None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.970578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cae687-fa20-4c05-a2ef-8dbe2f2cf6e6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:31.987472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:31.988019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:31.988302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:31.988690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:31.989353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:31.989701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:31.991133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.016728 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 343/691] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:32.022168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:32.041740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:32.042419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.042706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.042978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.063562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:32.065047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:32.092417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:32.093794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:32.094710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:32.094710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:32.094710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:32.094879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 349/692] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:32.103714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:32.135678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:32.238755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6d7f-8003-4f5c-9ad2-c18bceb8b531 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:32.273007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadc8f5-1cbb-4d0e-abf1-678ee877296d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:32.283425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:32.311555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:32.313440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.313732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.314009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.329920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:32.331421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:32.349640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:32.350672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:32.351615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 344/693] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:46:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:32.352198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:32.353075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.353075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.353236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.354026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:32.357084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:32.357084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.361075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:32.380468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 350/694] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:32.385665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:32.474462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cdcc9-d6dc-4523-bf65-660cf15a4242 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:32.480820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:32.509404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fb4aa-59ca-46ff-9d17-e08b624f450a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:32.512046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060545fd-fa7f-4929-b2fa-6aecb38b432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:32.512911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060545fd-fa7f-4929-b2fa-6aecb38b432d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.513392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060545fd-fa7f-4929-b2fa-6aecb38b432d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.513881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060545fd-fa7f-4929-b2fa-6aecb38b432d None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.514851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060545fd-fa7f-4929-b2fa-6aecb38b432d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:32.515562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060545fd-fa7f-4929-b2fa-6aecb38b432d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:32.517270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060545fd-fa7f-4929-b2fa-6aecb38b432d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.536817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 345/695] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:46:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:32.538405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739645f-6a22-442b-8d55-b035d5ff1efd None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.539641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739645f-6a22-442b-8d55-b035d5ff1efd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.539641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739645f-6a22-442b-8d55-b035d5ff1efd None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.541120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 346/696] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:32.542001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739645f-6a22-442b-8d55-b035d5ff1efd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:32.557381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c490f-5214-499e-b416-4f512741d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:32.558515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c490f-5214-499e-b416-4f512741d037 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.558629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c490f-5214-499e-b416-4f512741d037 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.558976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c490f-5214-499e-b416-4f512741d037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.567391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c490f-5214-499e-b416-4f512741d037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:32.569011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c490f-5214-499e-b416-4f512741d037 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8e3c490f-5214-499e-b416-4f512741d037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:32.581515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:32.582462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:32.583328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:32.587295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 351/697] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:32.588708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1476e8c-42a0-47a9-9ed2-4dddce1770c8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.589496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1476e8c-42a0-47a9-9ed2-4dddce1770c8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.589496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1476e8c-42a0-47a9-9ed2-4dddce1770c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.591734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:32.610926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c1476e8c-42a0-47a9-9ed2-4dddce1770c8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:32.612198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c1476e8c-42a0-47a9-9ed2-4dddce1770c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:32.654700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739645f-6a22-442b-8d55-b035d5ff1efd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:32.690984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59224409-ec49-4873-a339-844c86bd9585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:32.698378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 352/698] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:32.698503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224409-ec49-4873-a339-844c86bd9585 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.698503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224409-ec49-4873-a339-844c86bd9585 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.698503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59224409-ec49-4873-a339-844c86bd9585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.698761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1476e8c-42a0-47a9-9ed2-4dddce1770c8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:32.762921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:32.779352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224409-ec49-4873-a339-844c86bd9585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:32.781435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59224409-ec49-4873-a339-844c86bd9585 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.804673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1476e8c-42a0-47a9-9ed2-4dddce1770c8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:32.833675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:32.845410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c58705-e6ff-494e-855f-5d547b10e298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:32.846117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c58705-e6ff-494e-855f-5d547b10e298 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.846875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c58705-e6ff-494e-855f-5d547b10e298 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.847148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c58705-e6ff-494e-855f-5d547b10e298 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.848133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c58705-e6ff-494e-855f-5d547b10e298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:32.848441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c58705-e6ff-494e-855f-5d547b10e298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:32.850405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c58705-e6ff-494e-855f-5d547b10e298 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:32.870940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 353/699] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:32.870940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4904a-9f67-4d5a-b592-bb82587411ec None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.870940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4904a-9f67-4d5a-b592-bb82587411ec None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.870940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4904a-9f67-4d5a-b592-bb82587411ec None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.876139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 354/700] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:32.879822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4904a-9f67-4d5a-b592-bb82587411ec None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:32.911064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 347/701] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:32.912620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7735-56cc-4a15-ab6b-185b27ddc0de None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:32.912911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7735-56cc-4a15-ab6b-185b27ddc0de None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:32.913327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7735-56cc-4a15-ab6b-185b27ddc0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:32.932687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-436b7735-56cc-4a15-ab6b-185b27ddc0de None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:32.934023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-436b7735-56cc-4a15-ab6b-185b27ddc0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:33.005491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4904a-9f67-4d5a-b592-bb82587411ec None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.009947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 348/702] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:33.010747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b7735-56cc-4a15-ab6b-185b27ddc0de None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.029875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:33.040868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.042983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.042983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.042983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.042983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=c5e1b8292ac84020b6ac6db471a6e0ec) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:33.042983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.044426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.052238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 355/703] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:32 2022] GET /identity/v3/role_assignments?user.id=c5e1b8292ac84020b6ac6db471a6e0ec&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:33.053126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.098087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:33.134797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b7735-56cc-4a15-ab6b-185b27ddc0de None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.199569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.200616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.201095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.201514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.216882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.219441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.225591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e3ad7-91cf-4b0e-9b65-4606251a02b5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:33.253087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:33.254126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:33.255044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:33.255044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:33.255044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:33.256629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 349/704] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:33.262720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.275261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.275901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.276182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.276513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.293180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.294590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:33.313271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:33.314560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:33.315638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:33.315638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:33.315638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:33.315638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 356/705] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:33.315638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.371987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:33.395913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c422e15-2344-4e46-b940-4f3609ad7645 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.424241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:33.436167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.437243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.437243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.437478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.438067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:33.438402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.439776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.452242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 350/706] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:33.457883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.459813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77674e-80be-4154-9574-d3e4aa8493bc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.508944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.510208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.511090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.511574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.512409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.514547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:33.533196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:33.534299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:33.535156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:33.538436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 357/707] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:33.539508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.580143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cbfd2-dd34-4b2e-bf41-21c8c00a0c66 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.623996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.625030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.625383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.625731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.626499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:33.626873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.628322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.641311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 351/708] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:33.642406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.664463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c48ca-e10a-4ad5-aad0-c6544ca80313 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.668521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:33.694274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:33.702098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.703681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.704203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.704671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.713351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.715206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:33.731137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:33.732029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:33.732915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:33.735352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 358/709] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:33.736383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.773255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89939fc4-a25f-4347-8170-5c30cbae2600 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.818671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.819462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.819846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.820221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.821231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:33.822026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.823817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.830000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 352/710] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:33.831143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.872307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ac00e-2af8-4a26-a93b-299dc443dd4c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.910062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.910897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.911176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.911511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.916459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.916459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:33.948337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369b06-2a23-4fcc-b5f2-817f9a384131 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:33.956188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:33.982463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 359/711] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:46:33 2022] POST /identity/v3/users => generated 254 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:33.983626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:33.984279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1ddb9-661b-45a8-9464-659ff8a6435e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:33.984919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1ddb9-661b-45a8-9464-659ff8a6435e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:33.985283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1ddb9-661b-45a8-9464-659ff8a6435e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:33.985567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1ddb9-661b-45a8-9464-659ff8a6435e None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:33.986290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1ddb9-661b-45a8-9464-659ff8a6435e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:33.986652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1ddb9-661b-45a8-9464-659ff8a6435e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:33.988176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1ddb9-661b-45a8-9464-659ff8a6435e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:34.002851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 353/712] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:34.004348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44333a1a-5a17-44bc-89b6-69e34e85236c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:34.004649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44333a1a-5a17-44bc-89b6-69e34e85236c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:34.004929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44333a1a-5a17-44bc-89b6-69e34e85236c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:34.006575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 354/713] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:34.010805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44333a1a-5a17-44bc-89b6-69e34e85236c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:34.025430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:34.137584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eada87-810e-496f-8a69-fe930a6b2ced None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:34.143617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44333a1a-5a17-44bc-89b6-69e34e85236c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:34.189252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3967d-e206-4538-818c-fb335cd0d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:34.189875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3967d-e206-4538-818c-fb335cd0d91a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:34.190173 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3967d-e206-4538-818c-fb335cd0d91a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:34.190468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3967d-e206-4538-818c-fb335cd0d91a None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:34.191110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3967d-e206-4538-818c-fb335cd0d91a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:34.191414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3967d-e206-4538-818c-fb335cd0d91a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:34.192242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:34.192791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3967d-e206-4538-818c-fb335cd0d91a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:34.192887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:34.193122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:34.193393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:34.194153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7f5fe23bc8294bc3861e0db9dad2045e) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:34.194487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:34.197176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:34.201205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 360/714] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 22 10:46:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:34.203237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855be21-849e-494a-960b-84e297c1b2d1 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:34.203850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855be21-849e-494a-960b-84e297c1b2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:34.203850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855be21-849e-494a-960b-84e297c1b2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:34.212586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 355/715] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:34 2022] GET /identity/v3/role_assignments?user.id=7f5fe23bc8294bc3861e0db9dad2045e&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:34.232360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7855be21-849e-494a-960b-84e297c1b2d1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:34.234280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7855be21-849e-494a-960b-84e297c1b2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:34.325097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 361/716] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:34.634048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:34.676734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:34.768261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db41b4-19b0-49ac-bd41-4bd437713d89 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:34.808440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:34.809080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:34.809358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:34.809627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:34.810208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:34.811587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:34.822932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:34.824352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:34.825158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 356/717] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:34 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:34.830348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855be21-849e-494a-960b-84e297c1b2d1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:34.846378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:34.958196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855be21-849e-494a-960b-84e297c1b2d1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:34.992813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8d246-7e81-4345-b584-feea1414ae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:34.993425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8d246-7e81-4345-b584-feea1414ae35 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:34.993702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8d246-7e81-4345-b584-feea1414ae35 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:34.993972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8d246-7e81-4345-b584-feea1414ae35 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:34.994637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8d246-7e81-4345-b584-feea1414ae35 None admin] RBAC: Adding query filter params (name=network) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:34.994936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8d246-7e81-4345-b584-feea1414ae35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:34.996561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8d246-7e81-4345-b584-feea1414ae35 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:35.001425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 362/718] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:46:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:35.006974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:35.151243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:35.159188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9237e-b939-4743-bd08-f6eb7888cdbf None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:35.198603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778eea7-51f4-4c9f-95ba-30986c0580a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:35.199221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778eea7-51f4-4c9f-95ba-30986c0580a0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:35.199500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778eea7-51f4-4c9f-95ba-30986c0580a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:35.199771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778eea7-51f4-4c9f-95ba-30986c0580a0 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:35.200457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778eea7-51f4-4c9f-95ba-30986c0580a0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:35.200760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778eea7-51f4-4c9f-95ba-30986c0580a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:35.202143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778eea7-51f4-4c9f-95ba-30986c0580a0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:35.212145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 357/719] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:46:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:35.391379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c902f2-5347-4cd1-b6d4-793902a06218 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:35.391792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c902f2-5347-4cd1-b6d4-793902a06218 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:35.391999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c902f2-5347-4cd1-b6d4-793902a06218 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:35.393630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 363/720] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:35.399111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ac500-fde1-417d-be6e-b1d2b43cde75 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:35.399397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ac500-fde1-417d-be6e-b1d2b43cde75 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:35.399669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ac500-fde1-417d-be6e-b1d2b43cde75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:35.420105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-871ac500-fde1-417d-be6e-b1d2b43cde75 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:35.421855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-871ac500-fde1-417d-be6e-b1d2b43cde75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:35.505094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 358/721] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:35.915378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c902f2-5347-4cd1-b6d4-793902a06218 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:36.021668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:36.056007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c902f2-5347-4cd1-b6d4-793902a06218 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:36.113519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:36.114081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.114393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.114663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.126985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:36.131852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:36.132883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:36.133769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 364/722] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:36.139263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ac500-fde1-417d-be6e-b1d2b43cde75 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:36.243996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:36.257208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ac500-fde1-417d-be6e-b1d2b43cde75 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:36.292745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab1da6-7035-408c-b58f-7cd562ba3ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:36.293625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1da6-7035-408c-b58f-7cd562ba3ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.294084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1da6-7035-408c-b58f-7cd562ba3ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.294528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab1da6-7035-408c-b58f-7cd562ba3ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.295343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab1da6-7035-408c-b58f-7cd562ba3ba1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:36.295793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab1da6-7035-408c-b58f-7cd562ba3ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:36.297362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab1da6-7035-408c-b58f-7cd562ba3ba1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:36.306423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 359/723] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:36.312625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:36.441492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fd4e9-8709-4433-ae42-438939faed3c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:36.490429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:36.491331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.491770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.492191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.501452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:36.506548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:36.508214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:36.524149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c09e-0d6b-43f0-a3f3-75763005ff45 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.524782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c09e-0d6b-43f0-a3f3-75763005ff45 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.525263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c09e-0d6b-43f0-a3f3-75763005ff45 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:36.525376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:36.526348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:36.527224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:36.527224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:36.527224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:36.527224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 365/724] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:36.534355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:36.535739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 360/725] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:36.551127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce522b6-a09e-494b-acdb-15bc12b63647 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.551747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce522b6-a09e-494b-acdb-15bc12b63647 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.552235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce522b6-a09e-494b-acdb-15bc12b63647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.574304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-8ce522b6-a09e-494b-acdb-15bc12b63647 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:36.576082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8ce522b6-a09e-494b-acdb-15bc12b63647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:36.706430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 361/726] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:36.708051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e79ed-4b4e-4dd5-9c88-99bcc4c70fd3 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.708248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e79ed-4b4e-4dd5-9c88-99bcc4c70fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.708478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e79ed-4b4e-4dd5-9c88-99bcc4c70fd3 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.710097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 362/727] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:36.710968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9d4f3-8585-40dd-abdb-4eea49d79c76 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:36.715321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a28b40-88a0-40e2-a199-6e74bbf2dc59 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.715603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a28b40-88a0-40e2-a199-6e74bbf2dc59 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.715876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a28b40-88a0-40e2-a199-6e74bbf2dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.743805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-13a28b40-88a0-40e2-a199-6e74bbf2dc59 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:36.745267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-13a28b40-88a0-40e2-a199-6e74bbf2dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:36.775323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8a19d-a869-4357-ada4-58703469bf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:36.775939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8a19d-a869-4357-ada4-58703469bf65 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.776221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8a19d-a869-4357-ada4-58703469bf65 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.776493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8a19d-a869-4357-ada4-58703469bf65 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.778153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8a19d-a869-4357-ada4-58703469bf65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:36.778484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8a19d-a869-4357-ada4-58703469bf65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:36.782605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8a19d-a869-4357-ada4-58703469bf65 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:36.799936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 366/728] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:36.801308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202c24-7193-448e-823f-7a1a7cc1a8d1 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.801601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202c24-7193-448e-823f-7a1a7cc1a8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.801878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88202c24-7193-448e-823f-7a1a7cc1a8d1 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.807430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 367/729] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:36.808222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202c24-7193-448e-823f-7a1a7cc1a8d1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:36.838909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 363/730] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 22 10:46:36.846278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf588900-399b-4d59-a5f3-e84f12fcb0dd None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.846796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf588900-399b-4d59-a5f3-e84f12fcb0dd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.847713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf588900-399b-4d59-a5f3-e84f12fcb0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.872725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:36.881941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-bf588900-399b-4d59-a5f3-e84f12fcb0dd None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:36.888386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bf588900-399b-4d59-a5f3-e84f12fcb0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:36.890377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:36.943945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88202c24-7193-448e-823f-7a1a7cc1a8d1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:36.981022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:36.981820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:36.982108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:36.982428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:36.991072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:36.992619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:37.005721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:37.006734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:37.007682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:37.008222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 368/731] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:37.015253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.058575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 364/732] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:37.059508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf588900-399b-4d59-a5f3-e84f12fcb0dd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.176581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf588900-399b-4d59-a5f3-e84f12fcb0dd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.187428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe9d02-8349-453c-9a16-809436680a9b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.201447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:37.218389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.219375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.219666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.219941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.232376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.232748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:37.234022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.234562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.235043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.235990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:37.236546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:37.238245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:37.238619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:37.239515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:37.240343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:37.244644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 365/733] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:37.246646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.266010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 369/734] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:37.267371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.372104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44aa494-fae9-4ba9-8b8c-689f5144ecf0 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.385443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f14dc-e322-4d53-ac3e-4c6bb1339564 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.409555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.410181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.410489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.410754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.421720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:37.423819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.424425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.424703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.424975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.425635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:37.425935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:37.426323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:37.427210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:37.428621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:37.429267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:37.429423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 366/735] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:37.429423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.442501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 370/736] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:37.447217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.479695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:37.500764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:37.543544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a05427-50d5-4e61-8f8b-bfb7bc9e2752 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.573910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33406c-6fac-4010-8fa7-25fe1ee1885b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.584494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.585500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.585800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.586076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.611785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.612532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.612819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.613089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.613730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:37.615121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:37.658119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:37.659598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:37.679511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 371/737] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:46:37 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:37.680667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.730505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:37.788221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 367/738] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 22 10:46:37.789436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.803502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:37.829001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ab5e3-2b14-4b84-ba32-fc191fa282fa None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.873745 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf7eab-6ab5-487f-a559-46dd9b7192df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.875237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7eab-6ab5-487f-a559-46dd9b7192df None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.875237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7eab-6ab5-487f-a559-46dd9b7192df None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.875530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7eab-6ab5-487f-a559-46dd9b7192df None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.876612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf7eab-6ab5-487f-a559-46dd9b7192df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:37.876612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf7eab-6ab5-487f-a559-46dd9b7192df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:37.877790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf7eab-6ab5-487f-a559-46dd9b7192df None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:37.893228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 372/739] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:37.893228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626e82-344b-47f1-9fcc-2d59ab9af6c8 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.893228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626e82-344b-47f1-9fcc-2d59ab9af6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.893228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0626e82-344b-47f1-9fcc-2d59ab9af6c8 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.893228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 373/740] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:37.893678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626e82-344b-47f1-9fcc-2d59ab9af6c8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.908719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827b94b-9b21-4936-894f-9310c92708a5 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:37.945638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:37.946366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:37.946649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:37.946922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:37.958665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:37.960240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:37.976807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:37.977766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:37.983263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:37.983263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:37.983263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:37.983263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 368/741] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:37.983263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:37.996299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.011054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0626e82-344b-47f1-9fcc-2d59ab9af6c8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.050476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b08b4-3492-4876-84f6-6ba0df1459bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.051037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b08b4-3492-4876-84f6-6ba0df1459bb None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.051319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b08b4-3492-4876-84f6-6ba0df1459bb None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.051602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b08b4-3492-4876-84f6-6ba0df1459bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.065486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b08b4-3492-4876-84f6-6ba0df1459bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.067017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868b08b4-3492-4876-84f6-6ba0df1459bb None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.075733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-868b08b4-3492-4876-84f6-6ba0df1459bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:38.083473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:38.084622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:38.085479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:38.085479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:38.085479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:38.085479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 374/742] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:38.085944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06403-789d-444a-9dc1-d0756f4d871a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.086151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06403-789d-444a-9dc1-d0756f4d871a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.090881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06403-789d-444a-9dc1-d0756f4d871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.102614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9d8c5a-d6fb-4128-a051-5d0e7a8e557d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.105642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-79c06403-789d-444a-9dc1-d0756f4d871a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:38.107338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-79c06403-789d-444a-9dc1-d0756f4d871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:38.140572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.141306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.141623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.141888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.153083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.154564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:38.172049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:38.173021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:38.173916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:38.173916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:38.173916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:38.173916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 369/743] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:38.173916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.181579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 375/744] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 22 10:46:38.182305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06403-789d-444a-9dc1-d0756f4d871a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.256600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.287917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06403-789d-444a-9dc1-d0756f4d871a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.306105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37af8e-eeea-4ce4-8275-fad22d9ed02b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.320814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.326883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.327843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.328044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.328319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.328968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:38.329266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.330667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.346645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 376/745] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:38.347710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.351276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.353011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.353497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.353962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.354853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:38.355353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.356990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.375218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 370/746] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:38.377000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.478077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8d8de-6f48-491f-9f7e-ffe7b1b17e0d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.482728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de5540-9e93-4fdb-a33a-6b5e8d15835a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.505473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.517080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.517710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.517986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.518273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.522081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.522732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.523009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.523662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.524306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:38.524596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.524823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.525959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.526280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.538068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 377/747] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:38.538910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:38.539315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:38.540203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:38.541073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:38.543524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 371/748] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:38.544500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.598520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.649939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16271-be1a-4b9e-a2ee-f79220f818a1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.652763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87821c-10b1-4c9a-b981-349921959ed5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.688696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.689613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.690067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.690664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.690951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.691587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.691826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.691938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.692123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.693818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.699511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.700939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:38.713065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:38.718358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 378/749] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:38.719309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.772017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 372/750] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:46:38 2022] POST /identity/v3/services => generated 240 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:38.772860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.827280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936761-2947-4c1d-8188-3df88d9e55d8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.839803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.848182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:38.879254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.882409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.882409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.882409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.891096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.893025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:38.906614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:38.907653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:38.908543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:38.910865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 379/751] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:38.916858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:38.917003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e3f05-f9f5-47fb-85b4-591796a248bf None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:38.952603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:38.953282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:38.953565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:38.953831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:38.954823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:38.955038 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:38.956475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:38.970977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 373/752] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:38.971819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:39.027752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd15638-687c-4b7e-ba1e-8d16cd4a9a4c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.064637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.065245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.065532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.065783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.066632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:39.066934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.068369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.088191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 380/753] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:39.089139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:39.115888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f9178-0694-4d5f-a71d-79b28d6e0c68 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.119113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:39.120466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:39.153388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.154337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.155577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.156133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.157345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:39.157745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.159551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.179199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 374/754] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:46:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:39.180245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:39.221066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb5fa-2114-4a2e-98d8-775ae68220e3 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.265099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.265715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.265997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.266307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.267037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=c5e1b8292ac84020b6ac6db471a6e0ec) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:39.267345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.269311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.277297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 381/755] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:39 2022] GET /identity/v3/role_assignments?user.id=c5e1b8292ac84020b6ac6db471a6e0ec&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:39.279455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:39.312933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097feea-c296-4b8f-aa90-df0cca281781 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.361691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.363537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.364007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.364446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.427624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:39.469646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:39.481310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.484944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.489467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d201a-f79a-4b83-b97e-d2a455f1bd43 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.539233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.540111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.540574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.541016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.560097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.563590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:39.590395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:39.591654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:39.593178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:39.593178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:39.593178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:39.597498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 382/756] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:39.603544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:39.689448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 375/757] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:46:39.694385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:39.734020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cdbe6-48c7-49f5-b0ba-a90c284f58ac None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.772302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981cc51-29e5-428c-be68-14f23dee4f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.773428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981cc51-29e5-428c-be68-14f23dee4f8f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.773428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981cc51-29e5-428c-be68-14f23dee4f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.773866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981cc51-29e5-428c-be68-14f23dee4f8f None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.774657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981cc51-29e5-428c-be68-14f23dee4f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:39.775103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981cc51-29e5-428c-be68-14f23dee4f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.776646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981cc51-29e5-428c-be68-14f23dee4f8f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.784495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:39.790381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:39.794824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 383/758] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:39.796872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df5b1-deac-4ef8-af5a-66d68159cf88 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.797242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df5b1-deac-4ef8-af5a-66d68159cf88 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.797610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df5b1-deac-4ef8-af5a-66d68159cf88 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.803414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 384/759] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:39.804429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2df5b1-deac-4ef8-af5a-66d68159cf88 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:39.837317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876f639-fb20-4467-9555-835b415bb637 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.879131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3aaaf-9a17-4b22-8b1e-97c78aa15108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.879826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3aaaf-9a17-4b22-8b1e-97c78aa15108 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.880130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3aaaf-9a17-4b22-8b1e-97c78aa15108 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.880680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3aaaf-9a17-4b22-8b1e-97c78aa15108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.895733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3aaaf-9a17-4b22-8b1e-97c78aa15108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.897186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3aaaf-9a17-4b22-8b1e-97c78aa15108 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-10a3aaaf-9a17-4b22-8b1e-97c78aa15108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:39.915192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:39.916158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:39.917013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:39.917013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:39.917013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:39.920181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 376/760] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:39.921850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1bfa5-eef6-4fe6-ad10-81ac61b7efd2 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.922132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1bfa5-eef6-4fe6-ad10-81ac61b7efd2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.922423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f1bfa5-eef6-4fe6-ad10-81ac61b7efd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.930690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2df5b1-deac-4ef8-af5a-66d68159cf88 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:39.945136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a1f1bfa5-eef6-4fe6-ad10-81ac61b7efd2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:39.946450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a1f1bfa5-eef6-4fe6-ad10-81ac61b7efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:39.969989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:39.971053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:39.971557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:39.972075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:39.980073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:39.981812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:39.994793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:39.996005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:39.996911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:40.000141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 385/761] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:40.001450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.026334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 377/762] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:40.026334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1bfa5-eef6-4fe6-ad10-81ac61b7efd2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.086892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:40.092553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:40.129702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842a42-9902-49f2-a057-a1e01f6cca85 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.178490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.179538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.180029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.180708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:40.182141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:40.183934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:40.185528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:40.203865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 386/763] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:40.208659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.216149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f1bfa5-eef6-4fe6-ad10-81ac61b7efd2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.282186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.282825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.283103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.283376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:40.284070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:40.284376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:40.285799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:40.307773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 378/764] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:40.312372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.373337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8073ad-c989-4ceb-b872-0eda4c643203 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.396610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:40.413481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.414363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.414816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.415231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:40.422417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:40.424087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:40.436514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:40.437461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:40.438366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:40.440690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 387/765] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:40.441697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.463511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5dffd-7d73-459f-9ec9-82999abb038d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.467524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:40.501710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.502831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.503354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.503842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7f5fe23bc8294bc3861e0db9dad2045e/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:40.506153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=7f5fe23bc8294bc3861e0db9dad2045e, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:40.508021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:40.557874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e45a6b-2269-430d-bdff-fd6ff0dc43c5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.579136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 379/766] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:40 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7f5fe23bc8294bc3861e0db9dad2045e/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:40.580971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.599087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.599777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.600054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.600326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:40.601291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:40.603056 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:40.615240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:40.616398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:40.617266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 388/767] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:40.622458 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.724711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:40.755068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59fbce-1581-4e0b-9e87-f0a2e0455476 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.799081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.799704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.799981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.800249 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:40.800942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:40.801259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:40.803494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:40.814754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ea601-86d4-4f8d-8da0-d9d9902c2199 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.819446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 380/768] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:40.827170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.858190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.859072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.859505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.859905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:40.860665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:40.861091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:40.861744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:40.863301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:40.874439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 389/769] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:40.888148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:40.958921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339aae90-7683-40c1-b0bd-f25e10d4ef2d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:40.997754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696eac7b-128e-412e-92aa-1d5aed4aa703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:40.998458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696eac7b-128e-412e-92aa-1d5aed4aa703 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:40.998773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696eac7b-128e-412e-92aa-1d5aed4aa703 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:40.999041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696eac7b-128e-412e-92aa-1d5aed4aa703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.000112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696eac7b-128e-412e-92aa-1d5aed4aa703 None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=4047588f18044f6da1c9884d5e6ec0e2) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:41.000421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696eac7b-128e-412e-92aa-1d5aed4aa703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:41.002117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696eac7b-128e-412e-92aa-1d5aed4aa703 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:41.009465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 381/770] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/role_assignments?user.id=4047588f18044f6da1c9884d5e6ec0e2&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:41.026338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a8719-a4ed-4c85-a59c-87f4352994f6 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.026862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a8719-a4ed-4c85-a59c-87f4352994f6 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.027279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a8719-a4ed-4c85-a59c-87f4352994f6 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.031060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 382/771] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:41.035957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fa0bd-951f-4d0a-bf2f-70f9109cbf0b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.036453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fa0bd-951f-4d0a-bf2f-70f9109cbf0b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.036890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fa0bd-951f-4d0a-bf2f-70f9109cbf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.056452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-429fa0bd-951f-4d0a-bf2f-70f9109cbf0b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:41.057941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-429fa0bd-951f-4d0a-bf2f-70f9109cbf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:41.075991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef7d6c-9f12-40ee-bc64-c09c14cf2f0a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:41.113346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:41.121606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:41.122341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.122631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.122907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.124794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:41.125116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:41.127161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:41.134016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 390/772] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 22 10:46:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:41.144670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:41.145580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 383/773] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:41.418580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037be8-e341-4c53-9543-767b93035bb9 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:41.465713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:41.466532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.466908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.467276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.468639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] RBAC: Adding query filter params (interface=public, service_id=d9bd663ec07d458a930d2042a6265f80, region_id=RegionOne) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:41.469048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:41.470552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:41.478852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 391/774] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:46:41 2022] GET /identity/v3/endpoints?service_id=d9bd663ec07d458a930d2042a6265f80&interface=public®ion_id=RegionOne => generated 194 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:41.484317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fa0bd-951f-4d0a-bf2f-70f9109cbf0b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:41.490375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:41.523563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:41.564068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:41.626696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fa0bd-951f-4d0a-bf2f-70f9109cbf0b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:41.646503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90280bee-c2ca-40e8-b7fb-b36ac67e3e7c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:41.664413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f88657-6856-4935-b46f-b48f9b55effa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:41.665389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f88657-6856-4935-b46f-b48f9b55effa None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.665988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f88657-6856-4935-b46f-b48f9b55effa None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.666509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f88657-6856-4935-b46f-b48f9b55effa None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.667364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f88657-6856-4935-b46f-b48f9b55effa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:41.669016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f88657-6856-4935-b46f-b48f9b55effa None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:41.675917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 384/775] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:46:41 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:41.677621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5223949-725c-4ffa-8501-ea00afb14b90 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.678137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5223949-725c-4ffa-8501-ea00afb14b90 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.678656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5223949-725c-4ffa-8501-ea00afb14b90 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.680732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 385/776] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:41.686745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a99a-140b-422d-be55-e84615b9e0a7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.687402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a99a-140b-422d-be55-e84615b9e0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.687885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1a99a-140b-422d-be55-e84615b9e0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.690569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:41.691303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.691661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.692018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.692683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:41.694099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:41.706196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:41.707283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:41.708129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 392/777] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:41 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:41.715484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:41.719563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b9b1a99a-140b-422d-be55-e84615b9e0a7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:41.721124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b9b1a99a-140b-422d-be55-e84615b9e0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:41.872675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 386/778] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:41.895937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:41.899968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270311c-a53d-4566-ad20-bd7af352e2d0 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:41.940606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:41.941301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:41.941782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:41.942160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:41.942967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] RBAC: Adding query filter params (name=network) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:41.943389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:41.944852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:41.949936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 393/779] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:46:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:41.955769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a99a-140b-422d-be55-e84615b9e0a7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:41.978377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:42.100718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1a99a-140b-422d-be55-e84615b9e0a7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:42.145859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:42.146512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:42.146792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:42.147056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:42.147685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] RBAC: Adding query filter params (type=network) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:42.147980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:42.149335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:42.159145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 387/780] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:46:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:42.164567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:42.178721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:42.264594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:42.316121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48415-6a35-4b37-879d-fa2842b5526f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:42.327960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa5a55-4bbc-4cac-ba2a-35803b44895c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:42.366788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:42.367405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:42.367684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:42.367963 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:42.369164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] RBAC: Adding query filter params (interface=public, service_id=c7a3f9e9a5064b6a8179b28d9c42b703, region_id=RegionOne) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:42.369472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:42.371281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:42.378029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 394/781] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:46:42 2022] GET /identity/v3/endpoints?service_id=c7a3f9e9a5064b6a8179b28d9c42b703&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:42.383865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:42.398254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:42.398912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:42.399188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:42.399457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:42.422602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:42.427729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:42.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:42.429478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:42.432139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 388/782] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:42.433071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:42.437901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:42.548691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1942f58-ffb6-46b2-ac6c-15b19866c46b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:42.586607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:42.587221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:42.587502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:42.587777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:42.588390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:42.589874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:42.597399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 395/783] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:46:42 2022] GET /identity/v3/services => generated 772 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:42.658321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e710bc-92c9-475f-a004-86903bf28ac7 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:42.710926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:42.711549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:42.711831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:42.712104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:42.712770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:42.713069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:42.714466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:42.720059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:42.727176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 389/784] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:42.732576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:42.879810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:42.889318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6584931-8b5d-4a76-98db-21b8cf85a4b7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:42.925972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127f492-0fd6-44b5-bd25-7f43bae0df69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:42.926689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f492-0fd6-44b5-bd25-7f43bae0df69 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:42.926965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f492-0fd6-44b5-bd25-7f43bae0df69 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:42.927231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127f492-0fd6-44b5-bd25-7f43bae0df69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:42.937670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127f492-0fd6-44b5-bd25-7f43bae0df69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:42.939155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127f492-0fd6-44b5-bd25-7f43bae0df69 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-f127f492-0fd6-44b5-bd25-7f43bae0df69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:42.955088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:42.956034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:42.956890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:42.956890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:42.956890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:42.956890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 396/785] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:42.962196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:43.076160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ae79f-ddbf-4a78-81ac-fecaa699d3b6 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.076160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ae79f-ddbf-4a78-81ac-fecaa699d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.076160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ae79f-ddbf-4a78-81ac-fecaa699d3b6 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.076691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 397/786] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:43.081700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4906-ae01-4d4e-847e-fc7ccd61733e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.081980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4906-ae01-4d4e-847e-fc7ccd61733e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.082285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4906-ae01-4d4e-847e-fc7ccd61733e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.100868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e11e4906-ae01-4d4e-847e-fc7ccd61733e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:43.102063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e11e4906-ae01-4d4e-847e-fc7ccd61733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:43.192920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169099f-1e6a-441a-96dc-ed15ec0f1c55 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:43.198507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 398/787] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:43.262919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03194d3-fe72-4d25-a3e6-254a5429fa49 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.263325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03194d3-fe72-4d25-a3e6-254a5429fa49 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.263602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03194d3-fe72-4d25-a3e6-254a5429fa49 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.265553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 399/788] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:43.271695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37aa34-8355-4096-bdd6-54d953619c74 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.272077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37aa34-8355-4096-bdd6-54d953619c74 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.272444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37aa34-8355-4096-bdd6-54d953619c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.274940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:43.275572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.275857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.276132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.276786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:43.277087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:43.278478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:43.292054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2d37aa34-8355-4096-bdd6-54d953619c74 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:43.293394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2d37aa34-8355-4096-bdd6-54d953619c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:43.305377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 390/789] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:43.310713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:43.374372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 400/790] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:43.427763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:43.484994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dfc7a-f7d0-49f0-ac56-5f44dfaff652 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:43.534178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:43.534844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.535126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.535399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.549018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:43.550485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:43.567522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:43.568453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:43.569412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 391/791] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:43.575752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37aa34-8355-4096-bdd6-54d953619c74 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:43.576152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:43.579829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:43.708214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f224a9-70c0-4fe6-91ba-77f29e2796bd None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:43.725891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:43.745147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37aa34-8355-4096-bdd6-54d953619c74 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:43.745301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:43.745301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.745301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.745301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.745594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:43.745594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:43.746375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:43.765108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 392/792] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 22 10:46:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:43.766007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:43.792254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:43.792895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.793174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.793446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.804840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:43.809780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:43.810712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:43.812320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 401/793] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:43.813005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:43.878701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63807e8-c8b2-46ff-916f-22b431e8c076 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:43.901685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:43.914562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:43.915204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.915479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.915754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.927058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:43.930639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b785fa-a96b-4a9c-81c8-e1915c9144e5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:43.931803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:43.932751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:43.933571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 393/794] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:43.934790 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:43.970107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:43.970812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:43.971088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:43.971936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:43.972400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:43.983889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:43.985575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:44.002536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:44.003704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:44.004810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:44.004810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:44.004810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:44.011095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 402/795] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:44.012153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.038927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770acb-2729-4cab-8a68-561aabdae4c6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.074527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.074869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.075153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.075431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.076191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:44.076409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.078609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.087880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 394/796] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:44.088828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.125199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dba74-7947-4d0e-bcfd-7d6e091fbc43 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.143123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:44.165722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.166666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.167125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.167608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.168596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:44.169109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.170875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.180652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 403/797] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:44.181793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.197325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:44.202733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf4b52-ceff-4390-a568-7b7465221807 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.240577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.241426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.241848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.242282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.243184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:44.243646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.245442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.258123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 395/798] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:44.259148 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.291594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa962b44-ca75-45a8-8e11-53c2ca89df70 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.327672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.328308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.328585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.328854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.339888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.341321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:44.356897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:44.358686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:44.359816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:44.359816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:44.359816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:44.359816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 404/799] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:44.359816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.374888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270af39-bf74-4681-81e9-af13432d2602 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.392046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:44.414348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe50f8-a4b5-4e35-9c3d-c208b2727736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.414990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe50f8-a4b5-4e35-9c3d-c208b2727736 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.415265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe50f8-a4b5-4e35-9c3d-c208b2727736 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.415538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe50f8-a4b5-4e35-9c3d-c208b2727736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.427189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe50f8-a4b5-4e35-9c3d-c208b2727736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.428628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe50f8-a4b5-4e35-9c3d-c208b2727736 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-7cfe50f8-a4b5-4e35-9c3d-c208b2727736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:44.444586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:44.445542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:44.446449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:44.446449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:44.446449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:44.446449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 396/800] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:44.447205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f8fc3-0b1f-4a0c-a46f-6c2701842f10 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.447452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f8fc3-0b1f-4a0c-a46f-6c2701842f10 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.447724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f8fc3-0b1f-4a0c-a46f-6c2701842f10 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.450274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 397/801] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:44.450274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f8fc3-0b1f-4a0c-a46f-6c2701842f10 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.465811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:44.473208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99cebf-7f64-453f-be3d-b70dfa351c2a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.510883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.510883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.510883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.510883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.518174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.519641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:44.530975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:44.531884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:44.532798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 405/802] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:44.537445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.564258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f8fc3-0b1f-4a0c-a46f-6c2701842f10 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.601538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6277a-2c64-4e36-8955-99b81ca18f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.602452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6277a-2c64-4e36-8955-99b81ca18f4d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.602878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6277a-2c64-4e36-8955-99b81ca18f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.603294 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6277a-2c64-4e36-8955-99b81ca18f4d None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.604080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6277a-2c64-4e36-8955-99b81ca18f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:44.604525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6277a-2c64-4e36-8955-99b81ca18f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.606082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6277a-2c64-4e36-8955-99b81ca18f4d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.619984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 398/803] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:44.621398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199d8e-8257-4f85-9969-5d4a9a442cd3 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.621676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199d8e-8257-4f85-9969-5d4a9a442cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.621946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30199d8e-8257-4f85-9969-5d4a9a442cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.634895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:44.640776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-30199d8e-8257-4f85-9969-5d4a9a442cd3 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:44.641967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-30199d8e-8257-4f85-9969-5d4a9a442cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:44.644909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55663066-3636-4dd0-9bc2-22ea237eb322 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.679051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.679655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.679932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.680203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.680845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:44.681142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.682536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.695328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 406/804] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:44.696304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.713090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:44.726264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 399/805] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:44.726264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199d8e-8257-4f85-9969-5d4a9a442cd3 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.829220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14cf47-39d3-471f-9685-b9ef70efeee6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.844070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30199d8e-8257-4f85-9969-5d4a9a442cd3 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:44.879308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.880080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.880364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.880970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.881949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:44.882274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.884165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.892569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:44.893379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:44.893727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:44.894068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:44.899116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 407/806] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:44.899954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.907210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:44.908806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:44.917327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:44.922298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:44.923211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 400/807] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:44.924185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:44.989133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.021789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5302-c2af-46c1-bf20-010929633d9a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.031395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a27ab-cfdd-4d3e-8030-6a510d4da360 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.056575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52869af-2ed2-4a76-9840-00ed6188ead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.057201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52869af-2ed2-4a76-9840-00ed6188ead3 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.057480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52869af-2ed2-4a76-9840-00ed6188ead3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.057749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52869af-2ed2-4a76-9840-00ed6188ead3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.064730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.065388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.065504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52869af-2ed2-4a76-9840-00ed6188ead3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.065667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.065943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/c5e1b8292ac84020b6ac6db471a6e0ec/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.066915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52869af-2ed2-4a76-9840-00ed6188ead3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.067849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=c5e1b8292ac84020b6ac6db471a6e0ec, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.069257 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e52869af-2ed2-4a76-9840-00ed6188ead3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:45.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:45.078853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:45.079804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 408/808] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:46:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:45.080527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee52e-13de-4dc1-806d-1327805de664 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.080527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee52e-13de-4dc1-806d-1327805de664 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.080662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee52e-13de-4dc1-806d-1327805de664 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.081857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 409/809] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:45.082572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dee52e-13de-4dc1-806d-1327805de664 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.141653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 401/810] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:44 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/c5e1b8292ac84020b6ac6db471a6e0ec/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:45.142772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.155771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.195536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dee52e-13de-4dc1-806d-1327805de664 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.237117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.237736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.238013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.238428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.239164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:45.239577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.241034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.260845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 410/811] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:45.261836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.281933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a97298-eac3-42e4-a508-f65c36b4fc24 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.306628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.323865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885e18a-6689-4a83-91d3-577f4d0cb275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.324509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885e18a-6689-4a83-91d3-577f4d0cb275 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.324787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885e18a-6689-4a83-91d3-577f4d0cb275 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.325149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885e18a-6689-4a83-91d3-577f4d0cb275 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.325733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885e18a-6689-4a83-91d3-577f4d0cb275 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.327173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885e18a-6689-4a83-91d3-577f4d0cb275 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-5885e18a-6689-4a83-91d3-577f4d0cb275 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:45.337206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:45.338155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:45.339020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 402/812] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:45.339783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f520b4e-f50e-4551-9059-62f995d15807 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.339783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f520b4e-f50e-4551-9059-62f995d15807 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.340761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f520b4e-f50e-4551-9059-62f995d15807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.359265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5f520b4e-f50e-4551-9059-62f995d15807 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:45.361427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5f520b4e-f50e-4551-9059-62f995d15807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:45.379023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb5281-920b-4ee2-82ef-2bc33e1d5cdf None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.407972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.416489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.417107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.417386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.422335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.422335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:45.422335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.422335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.442545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 403/813] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:45.443351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f520b4e-f50e-4551-9059-62f995d15807 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.443908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 411/814] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:45.444785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.561631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b5888-6ff3-4e52-a5e7-230aacfedf94 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.569635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f520b4e-f50e-4551-9059-62f995d15807 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.584805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.605347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.606344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.606846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.607242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.607426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.608568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.608874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.609070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:45.609288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.609658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.611369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.621961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 412/815] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:45.623124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.676470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.685092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.687215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.729583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ec5e3-7d5c-4ad7-8bb2-4a17cbca933d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.767854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.768874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.769176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.769444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.783418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.785138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.792621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 404/816] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:46:45.794089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:45.803972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:45.804914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:45.805969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:45.805969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:45.805969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:45.805969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 413/817] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:45.806185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.815163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.905819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6233c8-4cf0-4c0d-a206-3ec0a660625c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.915689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3a9bb-fc3a-474b-93cc-1d849b1e2094 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:45.917996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:45.943575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.944211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.944488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.944757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.945381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:45.945678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.947069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.950714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:45.951499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:45.951841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:45.956630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] PATH_INFO: `/v3/services/network` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:45.956771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 405/818] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:45.956852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:45.956922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:45.956991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:45.965518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:45.966710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 22 10:46:45.968103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:45.968912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 414/819] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:45.968980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.041513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:46.064646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307884-218d-40ee-8bb0-b43fb886f45a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.093229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d69a2-6640-4f8a-8f03-6b632755b55b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.099771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.100690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.100975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.101242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.112732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.114126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.130114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.130767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:46.130884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:46.131731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:46.132602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:46.132602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:46.132602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:46.132765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.132765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.132765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:46.132947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 406/820] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:46.133606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.136636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.138039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.151377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 415/821] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:46.152212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.181411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:46.245230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd1b81-e76b-44e3-b34f-d6c5eff5ff3a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.271844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef92f0-9b41-4e27-8c26-139c36a27f29 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.274810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:46.280868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.281481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.281826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.282096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.286391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.286391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.308614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.309391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.309649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.309926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.310589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] RBAC: Adding query filter params (name=network) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:46.311451 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.313051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.322348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 416/822] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:46:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:46.322348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.359580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 407/823] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:46:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:46.360812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.448912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:46.463817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e760d91-b3b6-4e2e-9b57-ee5d98605fba None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.490866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf07d0e-6f3f-4511-ae76-e5b50e7f9cc8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.510631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.511675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.512510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.512960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.513780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:46.514848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.516954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.532912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 417/824] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:46.534165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.538746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.538746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.538746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.538746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.543792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.545279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:46.559895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:46.561639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:46.562581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:46.563009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 408/825] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:46.567027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.624128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:46.652617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2b0bc-71ac-4f29-ad1d-984cdccdcef8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.691793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80371e6-8947-4e4f-8899-9ebd082f72f2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.697177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.697796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.698071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.698378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.699068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] RBAC: Adding query filter params (type=network) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:46.699315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.701135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.710315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 418/826] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:46:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:46.711182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.737231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:46.737557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.738624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.738904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.739237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.748091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.752977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:46.766266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:46.767229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:46.768113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:46.775923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 409/827] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:46.775923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.840401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b156dd2-6a96-4a81-a286-179230ff486d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.888489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.889573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.890055 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.890574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.890679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c428d-5ece-4e05-b58e-e981fbba9ed2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:46.892171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:46.892453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:46.893620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.895277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:46.918245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 419/828] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:46.919467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:46.928184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:46.928802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:46.929086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:46.929362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:46.930027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:46.932323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:47.003108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:47.024471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 410/829] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:46:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:47.024471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:47.041216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfe16e-c84d-4fc9-a427-3ef802ea2b9b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:47.111936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e8a71-e66c-4e2d-91f7-1845e4556c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:47.112552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8a71-e66c-4e2d-91f7-1845e4556c08 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:47.112998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8a71-e66c-4e2d-91f7-1845e4556c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:47.113273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e8a71-e66c-4e2d-91f7-1845e4556c08 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:47.114158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e8a71-e66c-4e2d-91f7-1845e4556c08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:47.114495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e8a71-e66c-4e2d-91f7-1845e4556c08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:47.116030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e8a71-e66c-4e2d-91f7-1845e4556c08 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:47.131375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 420/830] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:47.137864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31e419-7b70-4054-8525-519e40ef16da None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:47.138212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31e419-7b70-4054-8525-519e40ef16da None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:47.142341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31e419-7b70-4054-8525-519e40ef16da None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:47.142341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 421/831] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:47.142341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31e419-7b70-4054-8525-519e40ef16da None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:47.189402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b0ebd-4786-4610-9d2a-919593137907 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:47.228538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d732f7-174e-4dc5-a17a-ef2ce9c75f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:47.229372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d732f7-174e-4dc5-a17a-ef2ce9c75f1e None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:47.229851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d732f7-174e-4dc5-a17a-ef2ce9c75f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:47.230223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d732f7-174e-4dc5-a17a-ef2ce9c75f1e None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/4047588f18044f6da1c9884d5e6ec0e2/roles/401c0ae4c0ad4b79bb877cbf745a71f2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:47.250524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d732f7-174e-4dc5-a17a-ef2ce9c75f1e None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=4047588f18044f6da1c9884d5e6ec0e2, role_id=401c0ae4c0ad4b79bb877cbf745a71f2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:47.252273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d732f7-174e-4dc5-a17a-ef2ce9c75f1e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:47.287520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31e419-7b70-4054-8525-519e40ef16da None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:47.315464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:47.321467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:47.322125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:47.322435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:47.322758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:47.323484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7f5fe23bc8294bc3861e0db9dad2045e) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:47.323843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:47.333541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 411/832] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:46:47 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/4047588f18044f6da1c9884d5e6ec0e2/roles/401c0ae4c0ad4b79bb877cbf745a71f2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:46:47.333752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:47.333878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a260c44-2e78-4ba5-9900-617cf4ffcd08 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:47.333878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a260c44-2e78-4ba5-9900-617cf4ffcd08 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:47.333878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a260c44-2e78-4ba5-9900-617cf4ffcd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:47.340134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 422/833] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:47 2022] GET /identity/v3/role_assignments?user.id=7f5fe23bc8294bc3861e0db9dad2045e&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:47.347099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3a260c44-2e78-4ba5-9900-617cf4ffcd08 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:47.350916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3a260c44-2e78-4ba5-9900-617cf4ffcd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:47.351045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:47.432699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 412/834] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:47.706581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:47.811535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e7f983-537e-440f-aafc-d9c9699e2f6d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:47.846409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:47.847044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:47.847323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:47.847595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:47.859993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:47.866103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:47.867079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:47.868683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 423/835] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:47.873649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a260c44-2e78-4ba5-9900-617cf4ffcd08 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:47.977711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a260c44-2e78-4ba5-9900-617cf4ffcd08 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:48.013097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:48.013741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.014182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.014539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.015194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:48.015491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:48.016853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:48.022166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:48.029992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 413/836] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:48.032746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:48.091560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:48.138246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9467740-c4d5-4ad7-9998-92412679d347 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:48.176380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:48.177013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.177290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.177562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.188807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:48.190272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:48.204454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:48.205356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:48.206686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:48.206686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:48.206686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:48.206686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 424/837] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:48.212869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:48.314156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647a9e8-2051-4e3a-8d4b-22ec949309d0 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:48.348576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06550622-5860-4843-90a9-e0cd342fd1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:48.349543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550622-5860-4843-90a9-e0cd342fd1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.350045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550622-5860-4843-90a9-e0cd342fd1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.350531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550622-5860-4843-90a9-e0cd342fd1e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.351366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06550622-5860-4843-90a9-e0cd342fd1e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:48.351858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06550622-5860-4843-90a9-e0cd342fd1e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:48.353477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06550622-5860-4843-90a9-e0cd342fd1e2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:48.370348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 414/838] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:48.372224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:48.429238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f74fb-cdfe-45ab-a156-ca91c8d28f39 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.429565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f74fb-cdfe-45ab-a156-ca91c8d28f39 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.429780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f74fb-cdfe-45ab-a156-ca91c8d28f39 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.431253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 415/839] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:48.436743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6618b1-ac5b-4caf-b664-1162e5fe39e8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.437022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6618b1-ac5b-4caf-b664-1162e5fe39e8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.437292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6618b1-ac5b-4caf-b664-1162e5fe39e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.456999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-7f6618b1-ac5b-4caf-b664-1162e5fe39e8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:48.458628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7f6618b1-ac5b-4caf-b664-1162e5fe39e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:48.485091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02239d1-fde8-4d02-b565-f4a7f6cbc66a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:48.492668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:48.520876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:48.526123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:48.527016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.527576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.528307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.536870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:48.538512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 416/840] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:48.538712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:48.551488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:48.552583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 22 10:46:48.553518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:48.556328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 425/841] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:46:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:48.564255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6618b1-ac5b-4caf-b664-1162e5fe39e8 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:48.668858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6618b1-ac5b-4caf-b664-1162e5fe39e8 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:48.704713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:48.705610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.706072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.706542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.707460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:48.707946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:48.709609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:48.727364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 417/842] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 22 10:46:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:48.734259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:48.805118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:48.840351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28305ec2-1d3d-4a5b-89a5-ed2f68f750c3 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:48.857868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:48.888968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:48.889817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.890096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.890391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.919193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04531fbc-db55-4c1a-8330-d6c51b9e0cd4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:48.930223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:48.960759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:48.961899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:48.962442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:48.962914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:48.963787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:48.965511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:48.972698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:48.974140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:48.975985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:48.976927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:48.977759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:48.986014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 418/843] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:48.993929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.103700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c8b23-3b49-4e9e-9ba7-9b0a0ff794ce None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:49.106310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 426/844] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:46:49.111427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.122032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:49.145456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:49.146459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.146960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.147445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.148314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:49.148819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:49.150432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:49.160541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 419/845] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:49.161720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.206380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:49.246764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c2725-b522-4fff-8491-9c5fd08382f9 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:49.291581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa71065-1e77-49ab-9601-d3ec1ef2f987 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:49.292859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:49.293490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.293762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.294302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.307270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:49.309325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:49.329570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:49.330680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:49.331768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:49.331768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:49.331768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:49.331942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 427/846] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:49.332986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7f860-67c0-49c2-a759-39011784f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:49.333750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f860-67c0-49c2-a759-39011784f32c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.334165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f860-67c0-49c2-a759-39011784f32c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.334580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7f860-67c0-49c2-a759-39011784f32c None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.335345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f860-67c0-49c2-a759-39011784f32c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:49.335773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f860-67c0-49c2-a759-39011784f32c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:49.337357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.337986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7f860-67c0-49c2-a759-39011784f32c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:49.350383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 420/847] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:49.410761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846089d-27d6-45cc-bd50-bf33014ec8c7 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.411049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846089d-27d6-45cc-bd50-bf33014ec8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.411240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846089d-27d6-45cc-bd50-bf33014ec8c7 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.412868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 421/848] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:49.418476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae5fe6-2c6e-4fbe-aa22-b0c80755fb84 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.418784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae5fe6-2c6e-4fbe-aa22-b0c80755fb84 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.419026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae5fe6-2c6e-4fbe-aa22-b0c80755fb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.439204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-70ae5fe6-2c6e-4fbe-aa22-b0c80755fb84 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:49.440492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-70ae5fe6-2c6e-4fbe-aa22-b0c80755fb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:49.452530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0819d-0f8c-441e-84fc-924b73829895 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:49.471696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:49.490742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be25b0-486d-4972-a911-6746df00450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:49.491445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be25b0-486d-4972-a911-6746df00450a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.491774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be25b0-486d-4972-a911-6746df00450a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.492052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be25b0-486d-4972-a911-6746df00450a None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.492704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be25b0-486d-4972-a911-6746df00450a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:49.493003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be25b0-486d-4972-a911-6746df00450a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:49.495087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be25b0-486d-4972-a911-6746df00450a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:49.507154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:49.514404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 428/849] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:49.515886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94743095-f5a2-4dbc-a03d-caa781a465d4 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.516179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94743095-f5a2-4dbc-a03d-caa781a465d4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.516456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94743095-f5a2-4dbc-a03d-caa781a465d4 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.517809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 429/850] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:49.518573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94743095-f5a2-4dbc-a03d-caa781a465d4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.537834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 422/851] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:49.539141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a7c0-308c-421f-9793-17a98876dc19 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.539429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a7c0-308c-421f-9793-17a98876dc19 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.539738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84a7c0-308c-421f-9793-17a98876dc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.559870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5f84a7c0-308c-421f-9793-17a98876dc19 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:49.561206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5f84a7c0-308c-421f-9793-17a98876dc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:49.638845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 423/852] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:49.647378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94743095-f5a2-4dbc-a03d-caa781a465d4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:49.683867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:49.684415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.684690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.684960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.692340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:49.693789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:49.704752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:49.705649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:49.706638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:49.710332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 430/853] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:49.711924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a7c0-308c-421f-9793-17a98876dc19 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.811781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.816463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:49.819761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84a7c0-308c-421f-9793-17a98876dc19 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:49.866753 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:49.876518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:49.877390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.877890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.878357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.879761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:49.879761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:49.881427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:49.898682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 424/854] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:49.900841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:49.937960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d22ab-3bf7-4a86-9a99-a19bd617e707 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:49.973990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:49.974656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:49.974937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:49.975315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:49.986108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:49.991322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:49.992236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:49.993065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 431/855] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:49.993739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.015844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00c171-9145-4b84-8edb-08b646bf55cb None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.053827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.054872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.055157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.055433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.056157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=c5e1b8292ac84020b6ac6db471a6e0ec) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:50.056469 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.057891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.074270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 425/856] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/role_assignments?user.id=c5e1b8292ac84020b6ac6db471a6e0ec&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:50.074270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.104413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:50.107256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:50.113721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee09ac-bff8-4855-91bd-001fb31ca48b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.167389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.168176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.168593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.168875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.169476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.172127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:50.186086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:50.187484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:50.188598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:50.190381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2d88-af49-4ca9-8eeb-580e376151af None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.192343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 432/857] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 22 10:46:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:50.193262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.229686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.230337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.230614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.230884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.231569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:50.231841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.233794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.243272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 426/858] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:50.250362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.324236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ecc8a-958d-41cf-a823-9a01f17ab8de None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.359944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.360578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.360763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:50.360866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.361679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.361787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:50.362036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.363435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.368384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 433/859] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:50.375841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.383248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e8f5f-d9d3-44af-b54d-8c43dc84bca6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.402470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:50.420878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.421766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.422048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.422345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.432998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.434606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:50.450309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:50.451289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:50.452137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:50.452137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:50.452137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:50.452137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 427/860] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:50.460479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.491772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbcdf5-f290-45c6-9df9-1b280a2f419e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.536845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.537828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.538471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.538750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.539490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:50.540125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.542105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.548552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 434/861] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:50.590326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f631d9-6952-46de-93e9-a144639a0650 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.629853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.630615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.631147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.631484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.632207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:50.632575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.634023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.642179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:50.648098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 428/862] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:50.655823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.753859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:50.765531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94bb0a-4fa3-45cf-9269-e8e628ba8ac6 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.804852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.805497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.805778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.806054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.814218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.815688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:50.827655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:50.828626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:50.829576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 435/863] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:46:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:50.835496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:50.952917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311f2e2-309a-4b1b-ac2d-f18c8581e7d8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:50.988029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:50.988746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:50.989231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:50.989503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:50.990247 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:50.990571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:50.992001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:51.010877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 429/864] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 22 10:46:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:51.017028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:51.124000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3d9bc-58c6-4a33-b6a2-439196580e3e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:51.160109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc29b1d-ef1c-4631-8fdf-9b3e64015754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:51.160724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29b1d-ef1c-4631-8fdf-9b3e64015754 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:51.161006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29b1d-ef1c-4631-8fdf-9b3e64015754 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:51.161279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc29b1d-ef1c-4631-8fdf-9b3e64015754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:51.180772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:51.233175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc29b1d-ef1c-4631-8fdf-9b3e64015754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:51.234664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc29b1d-ef1c-4631-8fdf-9b3e64015754 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:51.348707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 436/865] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:46:51.350377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:51.384513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8cae-aaf8-4ec8-8a6e-6666d7ebbd8b None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:51.385105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8cae-aaf8-4ec8-8a6e-6666d7ebbd8b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:51.385596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8cae-aaf8-4ec8-8a6e-6666d7ebbd8b None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:51.387418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 437/866] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:51.393822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef0cf-0746-404c-b0e3-aa367de26f9f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:51.394099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef0cf-0746-404c-b0e3-aa367de26f9f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:51.394397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef0cf-0746-404c-b0e3-aa367de26f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:51.413830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-18cef0cf-0746-404c-b0e3-aa367de26f9f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:51.415033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-18cef0cf-0746-404c-b0e3-aa367de26f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:51.417059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:51.464376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4c190-fcba-4426-b357-ace00ccdaa55 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:51.465604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:51.490556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 438/867] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:51.506037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:51.506709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:51.506988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:51.507262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:51.518020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:51.519538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:51.534137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:51.535108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:51.535984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:51.535984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:51.535984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:51.535984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 430/868] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:51.541060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cef0cf-0746-404c-b0e3-aa367de26f9f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:51.651996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cef0cf-0746-404c-b0e3-aa367de26f9f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:51.687961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:51.688585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:51.688866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:51.689143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:51.689799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:51.690103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:51.691585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:51.703777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 439/869] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:51.711357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:51.795441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:51.826859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccda41-6b30-4dd5-a9cb-b1ab7332cdc6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:51.838652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:51.862753 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:51.877627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:51.878300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:51.878589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:51.878866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:51.887494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:51.889633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:51.903919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:51.905481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:51.906393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:51.909488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 431/870] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:51.914327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:51.923418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2532e27-9f64-46e8-8d79-a81b85dde26b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:51.968576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:51.969638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:51.970425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:51.970888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:51.971741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:51.973743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:52.024926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d3180-8f1b-4b4d-a39d-5cf0cd7ee41e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:52.038275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 440/871] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:46:51 2022] POST /identity/v3/services => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:46:52.079074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e83f6-f456-4047-8898-9844d45a7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:52.079710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e83f6-f456-4047-8898-9844d45a7db7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.080030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e83f6-f456-4047-8898-9844d45a7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.080307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e83f6-f456-4047-8898-9844d45a7db7 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.081003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e83f6-f456-4047-8898-9844d45a7db7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:52.081303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e83f6-f456-4047-8898-9844d45a7db7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:52.082843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e83f6-f456-4047-8898-9844d45a7db7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:52.102107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 432/872] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:52.104188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:52.178506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263fcd3-274e-437a-98ba-b1d36296e68e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.178681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263fcd3-274e-437a-98ba-b1d36296e68e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.178984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263fcd3-274e-437a-98ba-b1d36296e68e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.180645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 433/873] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:52.187047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a04b20-7227-45c4-a8f7-ce1405f5978c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.187330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a04b20-7227-45c4-a8f7-ce1405f5978c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.187608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a04b20-7227-45c4-a8f7-ce1405f5978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.206952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-16a04b20-7227-45c4-a8f7-ce1405f5978c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:52.208157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-16a04b20-7227-45c4-a8f7-ce1405f5978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:52.225760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:52.228222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:52.231302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacef41-5968-4910-afbf-9ed6af272676 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:52.282256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e19ba8-f23c-466c-951c-1e658cf5e445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:52.282256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19ba8-f23c-466c-951c-1e658cf5e445 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.282256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19ba8-f23c-466c-951c-1e658cf5e445 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.282256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19ba8-f23c-466c-951c-1e658cf5e445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.282682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e19ba8-f23c-466c-951c-1e658cf5e445 None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=4047588f18044f6da1c9884d5e6ec0e2) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:52.282682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e19ba8-f23c-466c-951c-1e658cf5e445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:52.282682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e19ba8-f23c-466c-951c-1e658cf5e445 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:52.290128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 434/875] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:52.290363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 441/875] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:52 2022] GET /identity/v3/role_assignments?user.id=4047588f18044f6da1c9884d5e6ec0e2&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:52.568174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a04b20-7227-45c4-a8f7-ce1405f5978c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:52.671466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a04b20-7227-45c4-a8f7-ce1405f5978c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:52.707195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:52.707849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.708130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.708404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.708990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:52.710376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:52.715051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77116bd3-6cb5-4015-a9d5-a5e820b8c48f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.715338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77116bd3-6cb5-4015-a9d5-a5e820b8c48f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.715619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77116bd3-6cb5-4015-a9d5-a5e820b8c48f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.718143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 442/876] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:52.719752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:52.720734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:52.721574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 435/877] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:52.723008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910c80e-e27d-4ada-b3d7-36b2c3b793db None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.723293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910c80e-e27d-4ada-b3d7-36b2c3b793db None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.723575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910c80e-e27d-4ada-b3d7-36b2c3b793db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.726992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:52.742266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2910c80e-e27d-4ada-b3d7-36b2c3b793db None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:52.743473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2910c80e-e27d-4ada-b3d7-36b2c3b793db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:52.753704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:52.833993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 443/878] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:52.838684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e470147-8f7a-4c3e-be91-f708605ff215 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:52.875945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:52.876559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:52.876838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:52.877106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:52.877734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:52.878031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:52.879427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:52.884564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 436/879] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:46:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:52.890311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910c80e-e27d-4ada-b3d7-36b2c3b793db None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:52.988076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:52.996106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910c80e-e27d-4ada-b3d7-36b2c3b793db None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:53.033760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a7e83-cedf-483d-a532-5d36357a9ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:53.034396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e83-cedf-483d-a532-5d36357a9ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:53.034679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e83-cedf-483d-a532-5d36357a9ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:53.034952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7e83-cedf-483d-a532-5d36357a9ce6 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:53.035584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7e83-cedf-483d-a532-5d36357a9ce6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:53.035942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7e83-cedf-483d-a532-5d36357a9ce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:53.037308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7e83-cedf-483d-a532-5d36357a9ce6 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:53.042182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 444/880] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:46:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:53.121841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:53.217310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:53.235844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31908e-b63e-46f8-b975-a5fc742f2992 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:53.273751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f704023-0be3-4a2b-944b-94180985086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:53.274657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f704023-0be3-4a2b-944b-94180985086a None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:53.274938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f704023-0be3-4a2b-944b-94180985086a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:53.275209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f704023-0be3-4a2b-944b-94180985086a None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:53.275942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f704023-0be3-4a2b-944b-94180985086a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:53.277311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f704023-0be3-4a2b-944b-94180985086a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:53.355126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 437/881] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:46:53 2022] POST /identity/v3/services => generated 269 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:54.146576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3a3b1-fa4b-48bc-9079-4b98e2cca0c6 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:54.147073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3a3b1-fa4b-48bc-9079-4b98e2cca0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:54.147073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3a3b1-fa4b-48bc-9079-4b98e2cca0c6 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:54.148618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 445/882] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:54.154110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6014c2-0c93-4f6d-bddb-22dd9c26d973 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:54.155068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6014c2-0c93-4f6d-bddb-22dd9c26d973 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:54.155353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6014c2-0c93-4f6d-bddb-22dd9c26d973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:54.173920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-fc6014c2-0c93-4f6d-bddb-22dd9c26d973 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:54.175146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fc6014c2-0c93-4f6d-bddb-22dd9c26d973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:54.254066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 438/883] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:54.300389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a121-f643-47c1-82ff-379f5dafce09 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:54.301012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a121-f643-47c1-82ff-379f5dafce09 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:54.301481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a121-f643-47c1-82ff-379f5dafce09 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:54.303302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 446/884] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:54.308656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a93b2-0ad2-437d-93ab-710d1a9fd401 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:54.308950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a93b2-0ad2-437d-93ab-710d1a9fd401 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:54.309283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a93b2-0ad2-437d-93ab-710d1a9fd401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:54.338562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b32a93b2-0ad2-437d-93ab-710d1a9fd401 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:54.340129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b32a93b2-0ad2-437d-93ab-710d1a9fd401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:54.434437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 439/885] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:54.436129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:54.617020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a121-f643-47c1-82ff-379f5dafce09 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:54.764980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a93b2-0ad2-437d-93ab-710d1a9fd401 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:54.779399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a121-f643-47c1-82ff-379f5dafce09 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:54.829799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:54.830461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:54.830740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:54.831013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:54.831597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:54.832953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:54.843054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:54.844025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:54.844861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:54.849105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 447/886] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:54.850409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:54.882392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:54.971702 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a93b2-0ad2-437d-93ab-710d1a9fd401 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:54.981759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae991dd2-4abd-4440-b2fa-6bec1c5541d2 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.017727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.018731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.019194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.019644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.020086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.020782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.021137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.021483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.022188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:55.022685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.024245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.030290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 448/887] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:55.035624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.038618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:55.043580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:55.045140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:46:55.045955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:55.053225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 440/888] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 22 10:46:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:55.054356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.116292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:55.127114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:55.142581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea4043-7661-4416-9df1-dc3067046b2b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.163370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd26560-48e7-4c91-9036-61b30fdcc57b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.180034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557051-db61-4cb4-900d-262b70ee2609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.180664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557051-db61-4cb4-900d-262b70ee2609 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.180927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557051-db61-4cb4-900d-262b70ee2609 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.181188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557051-db61-4cb4-900d-262b70ee2609 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.181818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557051-db61-4cb4-900d-262b70ee2609 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:55.182118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557051-db61-4cb4-900d-262b70ee2609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.183738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98557051-db61-4cb4-900d-262b70ee2609 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.193886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 449/889] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:55.195391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba8eed-0a49-4e0b-99b0-5e2c7397b99f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.195681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba8eed-0a49-4e0b-99b0-5e2c7397b99f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.196018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba8eed-0a49-4e0b-99b0-5e2c7397b99f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.197238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 450/890] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:55.198446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba8eed-0a49-4e0b-99b0-5e2c7397b99f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.203655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6367d-2bd3-47ac-b43e-0bcbd378097d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.204786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6367d-2bd3-47ac-b43e-0bcbd378097d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.205063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6367d-2bd3-47ac-b43e-0bcbd378097d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.205329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6367d-2bd3-47ac-b43e-0bcbd378097d None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.205990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6367d-2bd3-47ac-b43e-0bcbd378097d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:55.206321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6367d-2bd3-47ac-b43e-0bcbd378097d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.207687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6367d-2bd3-47ac-b43e-0bcbd378097d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.215815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 441/891] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:55.217559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e716382-9f8d-4318-b6fc-05b91e781dac None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.217933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e716382-9f8d-4318-b6fc-05b91e781dac None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.218205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e716382-9f8d-4318-b6fc-05b91e781dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.236129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2e716382-9f8d-4318-b6fc-05b91e781dac None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:55.237319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2e716382-9f8d-4318-b6fc-05b91e781dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:55.313926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 442/892] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:55.314720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e716382-9f8d-4318-b6fc-05b91e781dac None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.327533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba8eed-0a49-4e0b-99b0-5e2c7397b99f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.366218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:55.366699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.367687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.368188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.368703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.370696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] RBAC: Adding query filter params (interface=public, service_id=044bb20221ae42bfb80ba31d2c8dbd9f, region_id=RegionOne) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:55.371229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.373433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.385139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 451/893] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/endpoints?service_id=044bb20221ae42bfb80ba31d2c8dbd9f&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:55.386687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.405383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:55.421318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e716382-9f8d-4318-b6fc-05b91e781dac None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.458658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3f03b2-a402-4881-9674-90d3df8edb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.459575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f03b2-a402-4881-9674-90d3df8edb06 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.459863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f03b2-a402-4881-9674-90d3df8edb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.460135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3f03b2-a402-4881-9674-90d3df8edb06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.470907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f03b2-a402-4881-9674-90d3df8edb06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.472369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3f03b2-a402-4881-9674-90d3df8edb06 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-0d3f03b2-a402-4881-9674-90d3df8edb06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:55.487526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:55.488489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:55.489377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:55.489377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:55.489377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:55.489377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 443/894] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:55.489899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b59a-0ccd-4b8a-8402-4c239602513c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.490057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b59a-0ccd-4b8a-8402-4c239602513c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.490357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b59a-0ccd-4b8a-8402-4c239602513c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.491735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 444/895] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:55.495753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b59a-0ccd-4b8a-8402-4c239602513c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.505850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba418a85-e3b2-43bc-a997-8923a532db25 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.544070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094aae6-7b0a-439c-9358-2a776acd24b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.544700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094aae6-7b0a-439c-9358-2a776acd24b8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.544986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094aae6-7b0a-439c-9358-2a776acd24b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.545350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094aae6-7b0a-439c-9358-2a776acd24b8 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.545944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094aae6-7b0a-439c-9358-2a776acd24b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.547341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094aae6-7b0a-439c-9358-2a776acd24b8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.553793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 452/896] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 10:46:55.555620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a391e7-f0ed-46b6-92f2-a8ee9c911c67 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.556461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a391e7-f0ed-46b6-92f2-a8ee9c911c67 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.557278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a391e7-f0ed-46b6-92f2-a8ee9c911c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.584388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d4a391e7-f0ed-46b6-92f2-a8ee9c911c67 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:55.586654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d4a391e7-f0ed-46b6-92f2-a8ee9c911c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:55.606263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b59a-0ccd-4b8a-8402-4c239602513c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.617483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:55.647260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.647888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.648178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.648752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.649597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:55.649955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.651390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.666285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 445/897] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:55.666285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.672723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 453/898] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:55.673828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a391e7-f0ed-46b6-92f2-a8ee9c911c67 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.687590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:55.809294 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e158b-a53d-46e2-b0ae-3a3b8162145f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.818042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a391e7-f0ed-46b6-92f2-a8ee9c911c67 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:55.854201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.855435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.856358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.856864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.858053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.860429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.861880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:55.862627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:55.862907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:55.863181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:55.878047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:55.880096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:46:55.892890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:46:55.895129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:55.895666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 454/899] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:55.900321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.927959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 446/900] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:46:55 2022] POST /identity/v3/services => generated 241 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:46:55.960396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:55.989394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:55.990993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:56.040794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f78c2-6fb8-4a8e-b8fc-3bdbd4a21f78 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.085086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d6f62-d6b2-437f-a908-14ce1f8db541 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.088605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.089337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.089625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.089901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.090672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:56.090979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.092525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.111964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 455/901] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:56.117247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.136340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.136988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.137322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.137600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.138802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.140826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:56.150637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:56.151570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:56.152458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:56.157318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 447/902] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 22 10:46:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:56.158182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.243098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885c615-8ed3-497b-afd6-9de5315a6921 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.252547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:56.272022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec84280-d825-4532-beeb-1218612a0950 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.278513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:56.282540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.283345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.283825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.284299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.308564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.309209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.309519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.309850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.311788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:56.312099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.313554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.323733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 448/903] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:56.324606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.357265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.359044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.436627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3e327-d256-4233-bfcd-f6201b30d860 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.473850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 456/904] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 10:46:56.474992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.477001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.477708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.478061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.478367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.479006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:56.479311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.480748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.491974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 449/905] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:56.493052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.507314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:56.507938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:56.583060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703e234-2994-43ac-9d64-6b2fdefb77ce None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.610923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63a3cb-c5f9-4c6f-b3ad-2346db322826 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.619281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.620450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.620933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.621350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.633538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.635260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:46:56.652201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:56.653384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:46:56.655153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:46:56.655153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:46:56.655153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:56.655713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.656344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.656678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.657008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.662104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] RBAC: Adding query filter params (interface=public, service_id=435dcf4f7faf4c2e9e80b31fb25ee86b, region_id=RegionOne) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:56.662104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.662104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.662760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 457/906] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:56.663778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.674206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 450/907] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/endpoints?service_id=435dcf4f7faf4c2e9e80b31fb25ee86b&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:56.675121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.762020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:56.773375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:56.792049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff94e87-bf3a-47d9-a562-62f6c26186cc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.799539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a314178-d22d-407d-bc35-cd2486605ddd None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:56.831995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.833020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.833448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.833893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.834816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:56.835123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:56.835453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.835859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:56.836206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:56.836543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:56.837212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:56.837580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.838726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:56.850253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 451/908] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 10:46:56.861525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 458/909] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:56.870672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:56.981234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149db98-c7a8-47e6-9f89-ee7567b46e06 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:57.023796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:57.024365 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:57.024640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:57.024911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:57.032487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:57.033915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:57.035888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:46:57.046297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:57.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:57.049873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 452/910] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:46:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:57.053169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:57.168145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589699d-4623-4fa7-bda1-7e2070e661da None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:57.205504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31f1b-20d8-49b8-80eb-9e821370cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:57.206534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31f1b-20d8-49b8-80eb-9e821370cba2 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:57.207016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31f1b-20d8-49b8-80eb-9e821370cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:57.207488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31f1b-20d8-49b8-80eb-9e821370cba2 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:57.208373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31f1b-20d8-49b8-80eb-9e821370cba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:57.208892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31f1b-20d8-49b8-80eb-9e821370cba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:57.210521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31f1b-20d8-49b8-80eb-9e821370cba2 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:57.212333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:57.223928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 459/911] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:46:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:57.230212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:57.340407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a77bf-262b-4c46-b3b6-eb55d7ca25a6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:57.377816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6b08d-3650-4bd8-8649-1ef6efaa6fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:57.378817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b08d-3650-4bd8-8649-1ef6efaa6fbb None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:57.379299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b08d-3650-4bd8-8649-1ef6efaa6fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:57.379768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6b08d-3650-4bd8-8649-1ef6efaa6fbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:57.380692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6b08d-3650-4bd8-8649-1ef6efaa6fbb None admin] RBAC: Adding query filter params (role.id=aa67480f49d64305ac7e9d005617826a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=4047588f18044f6da1c9884d5e6ec0e2) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:57.381246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6b08d-3650-4bd8-8649-1ef6efaa6fbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:57.382903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6b08d-3650-4bd8-8649-1ef6efaa6fbb None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:57.389727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 453/912] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:46:57 2022] GET /identity/v3/role_assignments?user.id=4047588f18044f6da1c9884d5e6ec0e2&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=aa67480f49d64305ac7e9d005617826a => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:57.565253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa541b3-0d83-4c9f-b85e-4e254a3cc6f6 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:57.565760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa541b3-0d83-4c9f-b85e-4e254a3cc6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:57.565867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa541b3-0d83-4c9f-b85e-4e254a3cc6f6 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:57.567399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 460/913] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:57.572594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc0337-1a17-47e8-ad40-be7585ccd488 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:57.572882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc0337-1a17-47e8-ad40-be7585ccd488 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:57.573159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc0337-1a17-47e8-ad40-be7585ccd488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:57.593750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6ccc0337-1a17-47e8-ad40-be7585ccd488 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:57.595247 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6ccc0337-1a17-47e8-ad40-be7585ccd488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:57.656267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:57.667356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 454/914] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:57.935684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa541b3-0d83-4c9f-b85e-4e254a3cc6f6 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:57.995135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169770b1-3d33-49cb-b2b2-58a71a358d9e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:57.995432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169770b1-3d33-49cb-b2b2-58a71a358d9e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:57.995842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169770b1-3d33-49cb-b2b2-58a71a358d9e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:57.998023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 455/915] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:58.005040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198b3ea-198c-40fd-9ddc-2b6210a42df7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.005710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198b3ea-198c-40fd-9ddc-2b6210a42df7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.006509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198b3ea-198c-40fd-9ddc-2b6210a42df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.032252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0198b3ea-198c-40fd-9ddc-2b6210a42df7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:58.034199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0198b3ea-198c-40fd-9ddc-2b6210a42df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:58.053490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa541b3-0d83-4c9f-b85e-4e254a3cc6f6 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:58.088681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:58.089307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.089583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.089849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.090519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:58.092548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:58.103877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:58.104853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 22 10:46:58.105658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:58.107571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 461/916] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:58.112513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:58.131903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 456/917] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:46:58.173728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:58.214367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae667e-54c3-4829-8355-3b36fecd6b85 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:58.248631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:58.249230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.249504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.249770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.250423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:58.250718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:58.252066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:58.257463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 462/918] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:58.263619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198b3ea-198c-40fd-9ddc-2b6210a42df7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:58.368541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198b3ea-198c-40fd-9ddc-2b6210a42df7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:58.394065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:58.404206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:58.407308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:58.408221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.408662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.409208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.410065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:58.410578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:58.412165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:58.418338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 457/919] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 22 10:46:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:58.424894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:58.525879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d8f84-6496-4949-9869-f120aedacc3b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:58.533903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f297a3-0d32-41a7-92a2-47ab3c846147 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:58.569832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:58.570602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.570792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.571062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.571689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:58.573043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:58.573550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:58.574401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.574896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.576045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.577058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:58.578635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:58.582367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:58.583358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:46:58.584182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:58.584915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:58.588504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 463/920] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:58.589887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:58.674673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 458/921] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:46:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:46:58.706392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b33149-fc74-46f1-a505-9b8f6f544763 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:58.755371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d5da7-8023-48b9-b630-fd8254b67278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:58.755985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d5da7-8023-48b9-b630-fd8254b67278 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.756260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d5da7-8023-48b9-b630-fd8254b67278 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.756529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d5da7-8023-48b9-b630-fd8254b67278 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.757166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d5da7-8023-48b9-b630-fd8254b67278 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:58.757464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d5da7-8023-48b9-b630-fd8254b67278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:58.758865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d5da7-8023-48b9-b630-fd8254b67278 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:58.765151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 464/922] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:46:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:58.770982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:58.821608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:58.892886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b740b1-6290-4205-8817-caae094c5a0b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:58.911810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212b25b-b76e-4c6c-81d2-d3ac5737a63c None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.912229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212b25b-b76e-4c6c-81d2-d3ac5737a63c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.912509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212b25b-b76e-4c6c-81d2-d3ac5737a63c None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.913841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 465/923] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:46:58.920735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7e001-f008-4c43-a71a-eb2d35c543f9 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.921098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7e001-f008-4c43-a71a-eb2d35c543f9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.921445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7e001-f008-4c43-a71a-eb2d35c543f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.930684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:58.931543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:58.932010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:58.932452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:58.933253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:58.933735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:58.935348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:58.940971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 459/924] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:46:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:58.942849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c3d7e001-f008-4c43-a71a-eb2d35c543f9 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:58.944121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c3d7e001-f008-4c43-a71a-eb2d35c543f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:58.946329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:58.990614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:59.026255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 466/925] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:59.060636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b62c8-47d7-4468-88e9-5fe7d6874d58 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:59.067852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:59.097222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:59.097835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.098110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.098464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.099168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] RBAC: Adding query filter params (interface=public, service_id=8a27670ce3de441cb2d0465248bef477, region_id=RegionOne) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:59.099619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:59.100987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:59.106695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 460/926] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:46:58 2022] GET /identity/v3/endpoints?service_id=8a27670ce3de441cb2d0465248bef477&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:46:59.112162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7e001-f008-4c43-a71a-eb2d35c543f9 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:59.222375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7e001-f008-4c43-a71a-eb2d35c543f9 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:59.260515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560b77f-42aa-4de6-b7a4-ec5bb0b768c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:59.261122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560b77f-42aa-4de6-b7a4-ec5bb0b768c3 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.261403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560b77f-42aa-4de6-b7a4-ec5bb0b768c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.261674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560b77f-42aa-4de6-b7a4-ec5bb0b768c3 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.262308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560b77f-42aa-4de6-b7a4-ec5bb0b768c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:59.264301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560b77f-42aa-4de6-b7a4-ec5bb0b768c3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:59.270734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 467/927] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:46:59 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 10:46:59.354101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:59.369112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d9cfc-878f-4fd7-b0f2-f43ba6856581 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.369695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d9cfc-878f-4fd7-b0f2-f43ba6856581 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.369929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d9cfc-878f-4fd7-b0f2-f43ba6856581 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.371757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 468/928] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:46:59.378571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe507e-717b-4a33-962a-4c882ddc4412 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.378857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe507e-717b-4a33-962a-4c882ddc4412 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.379134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe507e-717b-4a33-962a-4c882ddc4412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.404423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f4fe507e-717b-4a33-962a-4c882ddc4412 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:46:59.406129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f4fe507e-717b-4a33-962a-4c882ddc4412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:46:59.444070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:59.483022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa19465-801a-4042-a4e7-3ad335532656 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:59.486583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 469/929] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:46:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:46:59.531387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:59.532334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.532835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.533219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.534179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:59.536389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:59.550076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:59.551022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 22 10:46:59.551888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:46:59.557182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 461/930] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 22 10:46:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:46:59.559337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe507e-717b-4a33-962a-4c882ddc4412 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:59.662850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe507e-717b-4a33-962a-4c882ddc4412 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:59.699737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:59.701297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.701787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.702280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.703201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:59.703699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:59.705258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:59.710620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 470/931] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:59.719659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:59.757799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:59.759915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:59.800778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:46:59.835463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988c4c7-138c-4d00-964f-776b80c88525 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:59.889175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:59.890780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.891356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.891831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.892865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:46:59.893659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:46:59.895702 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:46:59.899029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e467299-bf49-4113-ae16-018e6fb4f363 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:46:59.908775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 462/932] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:46:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:46:59.910275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:46:59.937359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:46:59.937719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:46:59.937998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:46:59.938289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:46:59.949550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:46:59.955455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:46:59.956364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:46:59.957381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 471/933] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 22 10:46:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:46:59.962499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:00.044870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:00.050221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ab4ac-368e-400d-931a-0092d50c301e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:00.065532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:00.073764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b5810-a1b9-4c01-9317-53d4242ed4d7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:00.098013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:00.099084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.099550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.100786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.102186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:00.104370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:00.112236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:00.112955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.113327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.113695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.114564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:00.115079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:00.117287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:00.130257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 472/934] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 22 10:46:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:00.133392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:00.192549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 463/935] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:46:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:00.258452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c856947-d6d2-4a0c-9de3-1dfcf0a916dc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:00.305818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:00.306914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.307399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.307894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.320858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:00.322857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:00.338706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:00.339807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:00.340701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:00.340701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:00.340701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:00.343230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 473/936] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:00.348710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:00.465420 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8650ee8-8278-42bd-83c4-4441d2c568ba None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:00.502716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:00.503375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.503627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.503899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.504557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:00.504928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:00.506411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:00.524646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 464/937] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:00.526019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:00.530514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:00.634183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8517a1a-c89d-4d68-b9e4-323b488ff3ac None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:00.670009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde70db-87d0-4aeb-9ea9-6de1d7e1d271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:00.670770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde70db-87d0-4aeb-9ea9-6de1d7e1d271 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.670989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde70db-87d0-4aeb-9ea9-6de1d7e1d271 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.671208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde70db-87d0-4aeb-9ea9-6de1d7e1d271 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.679627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde70db-87d0-4aeb-9ea9-6de1d7e1d271 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:00.681072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde70db-87d0-4aeb-9ea9-6de1d7e1d271 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-fdde70db-87d0-4aeb-9ea9-6de1d7e1d271 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:47:00.692405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:00.693327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:00.694271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 474/938] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:47:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:00.699521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:00.722931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:00.755907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0007-a10d-4800-954e-fa46b44ded64 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.756551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0007-a10d-4800-954e-fa46b44ded64 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.757021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd0007-a10d-4800-954e-fa46b44ded64 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.758717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 475/939] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:00.764434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d35b-c7e5-4438-a0b8-c0a6b617be31 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.764715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d35b-c7e5-4438-a0b8-c0a6b617be31 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.765740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506d35b-c7e5-4438-a0b8-c0a6b617be31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.786365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7506d35b-c7e5-4438-a0b8-c0a6b617be31 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:00.787870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7506d35b-c7e5-4438-a0b8-c0a6b617be31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:00.814515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d5441-09b6-410d-b546-57c2141cda7d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:00.852933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:00.853864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:00.854448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:00.855036 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:00.856005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:00.856548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:00.858506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:00.865274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:00.886452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 476/940] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:00.894069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 465/941] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 22 10:47:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:00.899676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506d35b-c7e5-4438-a0b8-c0a6b617be31 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.005634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506d35b-c7e5-4438-a0b8-c0a6b617be31 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.042102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.043010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.043443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.043885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.056114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.057806 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:01.072973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:01.074154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:01.075335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:01.075335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:01.075335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.081616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 477/942] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:01.083116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.159839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:01.195154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8b5db-b6e7-4310-8da3-c2122ed93857 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.210211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.234849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.235634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.236222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.236501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.237155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:01.237459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.239307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.255838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 466/943] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:01.261466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.271350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:01.338276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f087089-747c-4ef3-ba95-72d18868f330 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.390045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.391275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.391727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.392399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.393640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.395608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.396155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad775e-b177-4603-a2f2-97c196f56c72 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.399435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING keystone.server.flask.common [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 10:47:01.427238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:01.432101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7898f-2621-48e3-b9ca-b9b9c12741af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.435415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7898f-2621-48e3-b9ca-b9b9c12741af None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.435415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7898f-2621-48e3-b9ca-b9b9c12741af None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.435415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7898f-2621-48e3-b9ca-b9b9c12741af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.440563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7898f-2621-48e3-b9ca-b9b9c12741af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.442047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7898f-2621-48e3-b9ca-b9b9c12741af None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.449241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.sql.core [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 10:47:01.449241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 10:47:01.449241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [parameters: {'id': 'a25de685ae724cab8f6debe1de9add0e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 10:47:01.449241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81530) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-ebe7898f-2621-48e3-b9ca-b9b9c12741af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:47:01.454196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:01.458333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 467/944] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe704d9-8391-4297-9f53-4795ebeeaff2 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.459206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe704d9-8391-4297-9f53-4795ebeeaff2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.459797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe704d9-8391-4297-9f53-4795ebeeaff2 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.462520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 468/945] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:01.463629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205b33-796b-4443-bf05-3b053276e0dd None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.463912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205b33-796b-4443-bf05-3b053276e0dd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.464181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205b33-796b-4443-bf05-3b053276e0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a25de685ae724cab8f6debe1de9add0e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a25de685ae724cab8f6debe1de9add0e. Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application conn.query(q) Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 10:47:01.470768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application result.read() Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 22 10:47:01.471546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application next(self.gen) Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application yield resource Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application next(self.gen) Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise self.value Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application session.commit() Aug 22 10:47:01.475083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self.session.flush() Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application compat.raise_( Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 10:47:01.478584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 10:47:01.480784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application conn.query(q) Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application result.read() Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 10:47:01.483986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [parameters: {'id': 'a25de685ae724cab8f6debe1de9add0e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:01.487104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:01.489626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a25de685ae724cab8f6debe1de9add0e. Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 478/946] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 22 10:47:01 2022] POST /identity/v3/projects => generated 262 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:01.492591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.495115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-37205b33-796b-4443-bf05-3b053276e0dd None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:01.495115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-37205b33-796b-4443-bf05-3b053276e0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:01.562287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 469/947] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:01.563825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205b33-796b-4443-bf05-3b053276e0dd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.585153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc3f8d-d320-4186-bec0-43e93b15ac4f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.587166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:01.623098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.623910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.624258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.624603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.625416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:01.625807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.627701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.645680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 479/948] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:01.655507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.699298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205b33-796b-4443-bf05-3b053276e0dd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.726252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:01.739242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.739933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.740213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.740559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.749513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.751146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.761989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc7e3-6ba9-449c-be52-07eeec0103d8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:47:01.765093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:01.765943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:01.766835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 470/949] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:01.778042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.801139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.802274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.802726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.803167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/4047588f18044f6da1c9884d5e6ec0e2/roles/aa67480f49d64305ac7e9d005617826a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.805818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=4047588f18044f6da1c9884d5e6ec0e2, role_id=aa67480f49d64305ac7e9d005617826a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.807443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.879345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 480/950] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:47:01 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/4047588f18044f6da1c9884d5e6ec0e2/roles/aa67480f49d64305ac7e9d005617826a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:47:01.881986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:01.890070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4fb85-4dc2-44b5-8ae4-b8d1d44142b8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:01.917185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:01.938053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:01.938843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:01.939134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:01.939410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:01.940172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:01.940453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:01.941946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:01.968556 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 471/951] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:47:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:02.008228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb40ca-89ba-4754-bac2-ee3788666e1a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:02.049562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1b393-1b81-407e-af09-d4a4d7341960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:02.050220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1b393-1b81-407e-af09-d4a4d7341960 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.050531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1b393-1b81-407e-af09-d4a4d7341960 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.050802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1b393-1b81-407e-af09-d4a4d7341960 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.051397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1b393-1b81-407e-af09-d4a4d7341960 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:02.052792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1b393-1b81-407e-af09-d4a4d7341960 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-86d1b393-1b81-407e-af09-d4a4d7341960 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:02.063547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:02.064934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:02.065852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:02.068534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 481/952] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:47:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:02.074701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:02.126541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:02.186110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580fe0e-cea6-4353-a934-d5e1fd838ca2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:02.209216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329d0c0-c52a-4093-b983-105b1eb91fad None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.209514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329d0c0-c52a-4093-b983-105b1eb91fad None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.209774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329d0c0-c52a-4093-b983-105b1eb91fad None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.211394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 482/953] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:47:02.216531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7a02f-e5ed-4dcd-9df2-d36d28530877 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.216834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7a02f-e5ed-4dcd-9df2-d36d28530877 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.217090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7a02f-e5ed-4dcd-9df2-d36d28530877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.223466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:02.224071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.224359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.224825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.225471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:02.225775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:02.228094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:02.235284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 472/954] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:47:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:02.241534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-cdf7a02f-e5ed-4dcd-9df2-d36d28530877 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:02.241534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cdf7a02f-e5ed-4dcd-9df2-d36d28530877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:02.242721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:02.316837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 483/955] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:02.364747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fedafd-aa92-4c50-895a-82f37f27698f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:02.374634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:02.402718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:02.404567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.405052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.405502 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.406329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:02.406804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:02.408342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:02.413958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 473/956] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:47:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:02.420240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7a02f-e5ed-4dcd-9df2-d36d28530877 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:02.439244 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:02.524462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7a02f-e5ed-4dcd-9df2-d36d28530877 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:02.558932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:02.560401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.560856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.561302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.562149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:02.563705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:02.605381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:02.635449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 484/957] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:47:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:47:02.736175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3942113-1a12-4aa8-8cc0-d88df865134c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:02.781054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:02.781994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.782579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.783055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.783812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:02.785373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:02.795769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:02.796812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:02.797629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:02.800049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 474/958] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:02.805920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:02.886429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:02.917756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69abc34-8e40-469f-906f-4de8acc356be None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:02.954310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6efc77-bdca-4d30-9207-44c54ba651d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:02.955097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6efc77-bdca-4d30-9207-44c54ba651d5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:02.955475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6efc77-bdca-4d30-9207-44c54ba651d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:02.955843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6efc77-bdca-4d30-9207-44c54ba651d5 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:02.956571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6efc77-bdca-4d30-9207-44c54ba651d5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:02.956971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6efc77-bdca-4d30-9207-44c54ba651d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:02.958474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6efc77-bdca-4d30-9207-44c54ba651d5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:02.964037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 485/959] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:47:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:02.969810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:03.002719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:03.077577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa749355-4d2a-45eb-bcb6-44c60615b42e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:03.115821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cde2b-877d-4eaf-a124-d77980c17d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:03.116737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cde2b-877d-4eaf-a124-d77980c17d09 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:03.117183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cde2b-877d-4eaf-a124-d77980c17d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:03.117597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cde2b-877d-4eaf-a124-d77980c17d09 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:03.118409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cde2b-877d-4eaf-a124-d77980c17d09 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:03.118849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cde2b-877d-4eaf-a124-d77980c17d09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:03.120614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cde2b-877d-4eaf-a124-d77980c17d09 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:03.126780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 475/960] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:47:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:03.921822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c2a99-5622-4600-b358-0f61f661ba0e None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:03.922817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c2a99-5622-4600-b358-0f61f661ba0e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:03.923292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c2a99-5622-4600-b358-0f61f661ba0e None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:03.925071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 486/961] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:03.930365 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89225cc-65fc-40d8-93e6-ad3636590aae None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:03.931142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89225cc-65fc-40d8-93e6-ad3636590aae None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:03.931626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89225cc-65fc-40d8-93e6-ad3636590aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:03.950604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c89225cc-65fc-40d8-93e6-ad3636590aae None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:03.952056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c89225cc-65fc-40d8-93e6-ad3636590aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:04.023268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 476/962] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:04.030252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dfd63-49df-4430-b6eb-0caad9348186 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.030626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dfd63-49df-4430-b6eb-0caad9348186 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.030970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8dfd63-49df-4430-b6eb-0caad9348186 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.032593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 487/963] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:04.037724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fdac0-b6b1-45d3-ad84-bc3c1599c46f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.038166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fdac0-b6b1-45d3-ad84-bc3c1599c46f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.038614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86fdac0-b6b1-45d3-ad84-bc3c1599c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.061360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a86fdac0-b6b1-45d3-ad84-bc3c1599c46f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:04.062574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a86fdac0-b6b1-45d3-ad84-bc3c1599c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:04.081105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:04.132898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 477/964] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:04.300663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dfd63-49df-4430-b6eb-0caad9348186 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:04.404308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8dfd63-49df-4430-b6eb-0caad9348186 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:04.407445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fdac0-b6b1-45d3-ad84-bc3c1599c46f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:04.445981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:04.446635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.446913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.447185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.450940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:04.462385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:04.468590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:04.469497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:04.470759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 488/965] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 22 10:47:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:04.475722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:04.523525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86fdac0-b6b1-45d3-ad84-bc3c1599c46f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:04.561723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:04.562868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.563373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.563902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.564726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:04.566483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:04.569566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING keystone.server.flask.common [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 22 10:47:04.578794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141442f-6245-42ab-9a1b-eb48797fb535 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:04.609368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.sql.core [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 10:47:04.609368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 10:47:04.609368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [parameters: {'user_id': '22cc7cb69871446a8d104f45953e5b77', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 10:47:04.609368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81529) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 22 10:47:04.614035 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:04.614669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.614993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.615264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.615925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:04.616228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:04.617623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application conn.query(q) Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result.read() Aug 22 10:47:04.619610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 10:47:04.620496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application next(self.gen) Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application yield resource Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application next(self.gen) Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.session.rollback() Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.force_reraise() Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise self.value Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application session.commit() Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 22 10:47:04.621440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._prepare_impl() Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.session.flush() Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._flush(objects) Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application compat.raise_( Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application flush_context.execute() Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rec.execute(self) Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 22 10:47:04.622942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 22 10:47:04.630768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result = self._query(query) Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application conn.query(q) Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application result.read() Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 22 10:47:04.633006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [parameters: {'user_id': '22cc7cb69871446a8d104f45953e5b77', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:04.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 22 10:47:04.637549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 478/966] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 22 10:47:04 2022] POST /identity/v3/users => generated 160 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:04.639899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:04.641778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 489/967] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 22 10:47:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:04.644111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:04.663070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:04.707449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:04.749228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08740c10-d02f-4399-9b93-8978c9a60bd7 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:04.751172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c2b59-f865-400d-8e1d-8581ed71d9ac None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:04.786070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3906d3b-37d7-4186-8bbc-9816c4d29335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:04.786856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3906d3b-37d7-4186-8bbc-9816c4d29335 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.787227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3906d3b-37d7-4186-8bbc-9816c4d29335 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.787593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3906d3b-37d7-4186-8bbc-9816c4d29335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.787975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403e0eb-f318-4159-82d3-6517f4606597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:04.788649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403e0eb-f318-4159-82d3-6517f4606597 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.788928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403e0eb-f318-4159-82d3-6517f4606597 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.789194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403e0eb-f318-4159-82d3-6517f4606597 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.797047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403e0eb-f318-4159-82d3-6517f4606597 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:04.798556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3906d3b-37d7-4186-8bbc-9816c4d29335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:04.798691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403e0eb-f318-4159-82d3-6517f4606597 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:04.800158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3906d3b-37d7-4186-8bbc-9816c4d29335 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-a403e0eb-f318-4159-82d3-6517f4606597 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:47:04.811804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:04.812956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:04.813876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 479/968] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:47:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:04.814852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d715-1c27-4e57-8576-a1d822a7ea21 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.815032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d715-1c27-4e57-8576-a1d822a7ea21 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.815260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20d715-1c27-4e57-8576-a1d822a7ea21 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.822417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 480/969] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-c3906d3b-37d7-4186-8bbc-9816c4d29335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:04.822551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:04.823431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:04.824376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:04.824376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:04.824376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:04.824542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d715-1c27-4e57-8576-a1d822a7ea21 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:04.824618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 490/970] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:04.825643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142d25-bd85-45b0-b658-ada49e369b22 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:04.825923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142d25-bd85-45b0-b658-ada49e369b22 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:04.826568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da142d25-bd85-45b0-b658-ada49e369b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:04.844554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-da142d25-bd85-45b0-b658-ada49e369b22 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:04.845731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-da142d25-bd85-45b0-b658-ada49e369b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:04.920483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 491/971] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:04.921350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da142d25-bd85-45b0-b658-ada49e369b22 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:04.928373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:04.962614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20d715-1c27-4e57-8576-a1d822a7ea21 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:04.968347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:05.000160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8c4e-ce42-468a-a456-c01bb62940cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.000776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8c4e-ce42-468a-a456-c01bb62940cf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.001053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8c4e-ce42-468a-a456-c01bb62940cf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.001371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8c4e-ce42-468a-a456-c01bb62940cf None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.002071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8c4e-ce42-468a-a456-c01bb62940cf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:05.002401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8c4e-ce42-468a-a456-c01bb62940cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.004525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8c4e-ce42-468a-a456-c01bb62940cf None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.042678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da142d25-bd85-45b0-b658-ada49e369b22 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.049771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 481/972] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 22 10:47:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:05.065970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5d5b-a26b-41db-ab80-1432197228ee None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.066335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5d5b-a26b-41db-ab80-1432197228ee None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.066549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5d5b-a26b-41db-ab80-1432197228ee None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.068226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 482/973] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:05.075897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46f472-a14c-4ba8-8f33-cff4f0c7bcb2 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.076407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46f472-a14c-4ba8-8f33-cff4f0c7bcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.077323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46f472-a14c-4ba8-8f33-cff4f0c7bcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.090520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.092119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.092551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.092985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.093805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:05.094300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.096865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.102382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2f46f472-a14c-4ba8-8f33-cff4f0c7bcb2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:05.104295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2f46f472-a14c-4ba8-8f33-cff4f0c7bcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:05.112105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 492/974] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:05.118266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.208128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 483/975] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:05.211136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46f472-a14c-4ba8-8f33-cff4f0c7bcb2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.256414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:05.265031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4a3d1-8f68-4d42-bc63-8a41240a00ec None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.285912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:05.310711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.311355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.311626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.311897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.321179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.322722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:47:05.334997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:05.336154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:05.337280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 493/976] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:05.337914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46f472-a14c-4ba8-8f33-cff4f0c7bcb2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.346300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.381442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.382398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.382681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.382955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.383538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.384899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:05.394401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:05.395985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:05.397460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 484/977] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:05.402140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.464725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89628e4-7271-4dd2-8179-96dfbd2a6b69 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.500880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.501480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.501757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.502028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.502779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:05.503081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.504489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.513244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf0991-6bc1-4bc1-81f2-8985c5a4fc6c None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.521883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 494/978] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:05.522702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.539637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:05.543175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:05.548010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.548618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.548893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.549160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.549778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:05.550074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.551463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.556095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 485/979] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:05.556862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.628265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24814-3ed6-4a14-bf41-84b5494458da None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.656486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c82e7-0deb-4194-ac68-eaa9f596147b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.661492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.662100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.662410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.662690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.663327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.664675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.689282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.689883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.690160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.690456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.746825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:05.756031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.757436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.763853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 495/980] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:47:05 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:05.764832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.858652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:05.889406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 486/981] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:05.890709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.895860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd05f-bfcd-4a8e-a5d5-6a092f0df711 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:05.936291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:05.936926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:05.937515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:05.937515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:05.938277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:05.939098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:05.939941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:05.946501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 496/982] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:05.953600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:05.992646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:06.002725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad47bb-fd9a-455b-9dd2-93f0509ecd3d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:06.036799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:06.037434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:06.037710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:06.037977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:06.047870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:06.049297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:06.064138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:06.065098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:06.065980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:06.065980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:06.065980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:06.065980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 487/983] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:06.070634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:06.071081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69692e51-076d-418d-b0e2-b351b26ddc9c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:06.109802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:06.110833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:06.111264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:06.111686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:06.112689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] RBAC: Adding query filter params (interface=admin, service_id=8a27670ce3de441cb2d0465248bef477, region_id=RegionOne) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:06.113144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:06.114800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:06.121146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 497/984] 10.222.0.48 () {56 vars in 1249 bytes} [Mon Aug 22 10:47:05 2022] GET /identity/v3/endpoints?service_id=8a27670ce3de441cb2d0465248bef477&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:06.127655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:06.130358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:06.176805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c41f-7165-476d-98d6-b76c11181d2e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:06.213734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:06.214675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:06.215141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:06.215535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:06.216254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:06.216549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:06.218136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:06.233709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 488/985] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:06.238270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:06.241166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06382d62-918e-448f-9884-856962b1c7a0 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:06.241646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:06.277675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:06.278301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:06.278584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:06.278861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:06.279470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:06.280838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:06.287483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 498/986] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:47:06 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 10:47:06.359360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd6f3f-0a73-49ae-87d4-8024c684b6a2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:06.399002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:06.399663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:06.399942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:06.400215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:06.408756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:06.410243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:47:06.423693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:06.424712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:06.430577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 489/987] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:06.431100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:06.502256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:06.538109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73a2f6-9cb4-4821-9105-633967b31090 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:06.570715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5240d08-92a9-408f-b20a-86b6d560cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:06.571314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5240d08-92a9-408f-b20a-86b6d560cf38 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:06.571588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5240d08-92a9-408f-b20a-86b6d560cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:06.571851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5240d08-92a9-408f-b20a-86b6d560cf38 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:06.572531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5240d08-92a9-408f-b20a-86b6d560cf38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:06.572886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5240d08-92a9-408f-b20a-86b6d560cf38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:06.574303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5240d08-92a9-408f-b20a-86b6d560cf38 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:06.586617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 499/988] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:06.593062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:06.655179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:06.694327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da907b6-82a6-42b2-9359-ca291fbc73b4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:06.727025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffa5ce-d808-4d81-bd21-c527620c1158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:06.727632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa5ce-d808-4d81-bd21-c527620c1158 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:06.727909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa5ce-d808-4d81-bd21-c527620c1158 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:06.728177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa5ce-d808-4d81-bd21-c527620c1158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:06.728894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffa5ce-d808-4d81-bd21-c527620c1158 None admin] RBAC: Adding query filter params (role.id=aa67480f49d64305ac7e9d005617826a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=4047588f18044f6da1c9884d5e6ec0e2) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:06.729196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffa5ce-d808-4d81-bd21-c527620c1158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:06.730643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffa5ce-d808-4d81-bd21-c527620c1158 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:06.742353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 490/989] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:06 2022] GET /identity/v3/role_assignments?user.id=4047588f18044f6da1c9884d5e6ec0e2&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=aa67480f49d64305ac7e9d005617826a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:07.720707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fa914-e4c5-4b28-baef-47c8c74263b0 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:07.721261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fa914-e4c5-4b28-baef-47c8c74263b0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:07.721261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fa914-e4c5-4b28-baef-47c8c74263b0 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:07.722756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 500/990] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:07.727749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09f45c-80c5-4a7c-9971-8a6b53bd112f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:07.728140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09f45c-80c5-4a7c-9971-8a6b53bd112f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:07.728312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd09f45c-80c5-4a7c-9971-8a6b53bd112f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:07.747602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-cd09f45c-80c5-4a7c-9971-8a6b53bd112f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:07.748796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cd09f45c-80c5-4a7c-9971-8a6b53bd112f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:07.817034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 491/991] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:08.086657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fa914-e4c5-4b28-baef-47c8c74263b0 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:08.190598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fa914-e4c5-4b28-baef-47c8c74263b0 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:08.216068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:08.225113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29551218-ed2d-4146-8337-f55b73727f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:08.225752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29551218-ed2d-4146-8337-f55b73727f67 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.226030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29551218-ed2d-4146-8337-f55b73727f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.226339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29551218-ed2d-4146-8337-f55b73727f67 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.226928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29551218-ed2d-4146-8337-f55b73727f67 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:08.228353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29551218-ed2d-4146-8337-f55b73727f67 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-29551218-ed2d-4146-8337-f55b73727f67 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:08.237545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:08.239419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:08.240279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:08.241606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 501/992] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:08.244827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09f45c-80c5-4a7c-9971-8a6b53bd112f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:08.255988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9e96b-c5d8-4b73-a225-721f73273a49 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.256281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9e96b-c5d8-4b73-a225-721f73273a49 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.256547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9e96b-c5d8-4b73-a225-721f73273a49 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.258190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 502/993] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:47:08.263508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d905e-6b61-44be-8b3a-be895d7a2d90 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.263786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d905e-6b61-44be-8b3a-be895d7a2d90 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.264172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d905e-6b61-44be-8b3a-be895d7a2d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.283748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-431d905e-6b61-44be-8b3a-be895d7a2d90 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:08.284934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-431d905e-6b61-44be-8b3a-be895d7a2d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:08.305074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:08.350954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd09f45c-80c5-4a7c-9971-8a6b53bd112f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:08.362917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 503/994] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:08.389953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:08.390683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.391040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.391387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.392105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:08.392478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:08.393936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:08.399027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 492/995] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:47:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:08.405078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d905e-6b61-44be-8b3a-be895d7a2d90 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:08.506633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d905e-6b61-44be-8b3a-be895d7a2d90 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:08.527785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:08.542168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:08.542809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.543093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.543412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.544145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:08.544445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:08.545815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:08.555663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 504/996] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:47:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:08.556630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:08.633875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:08.660215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c1da0-fd35-4322-9303-0c57119aed04 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:08.683280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:08.694986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad1ead-db40-41a8-9677-6ea4ce05305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:08.695599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad1ead-db40-41a8-9677-6ea4ce05305c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.695878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad1ead-db40-41a8-9677-6ea4ce05305c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.696149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad1ead-db40-41a8-9677-6ea4ce05305c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.696837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad1ead-db40-41a8-9677-6ea4ce05305c None admin] RBAC: Adding query filter params (interface=public, service_id=d14041f66c594cb696dd6281f6be826e, region_id=RegionOne) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:08.697139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad1ead-db40-41a8-9677-6ea4ce05305c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:08.698529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad1ead-db40-41a8-9677-6ea4ce05305c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:08.703953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 493/997] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:47:08 2022] GET /identity/v3/endpoints?service_id=d14041f66c594cb696dd6281f6be826e&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:08.705087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b618b-7f11-44b2-8edc-bae2bcad3388 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.705371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b618b-7f11-44b2-8edc-bae2bcad3388 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.705643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b618b-7f11-44b2-8edc-bae2bcad3388 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.706845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 494/998] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:08.711494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b618b-7f11-44b2-8edc-bae2bcad3388 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:08.737900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0052899-3c2d-4576-94ec-cf7497085027 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:08.771455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b942cb-f035-41c9-8d83-26ccd3de2f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:08.772088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b942cb-f035-41c9-8d83-26ccd3de2f0a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.772377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b942cb-f035-41c9-8d83-26ccd3de2f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.772691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b942cb-f035-41c9-8d83-26ccd3de2f0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.773284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b942cb-f035-41c9-8d83-26ccd3de2f0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:08.774711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b942cb-f035-41c9-8d83-26ccd3de2f0a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-32b942cb-f035-41c9-8d83-26ccd3de2f0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:08.784349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:08.785363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:08.786211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 505/999] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 22 10:47:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:08.787056 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398f76-0461-435c-8d6e-2b5f28759c82 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.787056 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398f76-0461-435c-8d6e-2b5f28759c82 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.787193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4398f76-0461-435c-8d6e-2b5f28759c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.804954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d4398f76-0461-435c-8d6e-2b5f28759c82 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:08.806146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d4398f76-0461-435c-8d6e-2b5f28759c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:08.817517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b618b-7f11-44b2-8edc-bae2bcad3388 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:08.851384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:08.851555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:08.851995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:08.852413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:08.852534 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:08.855752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:08.855752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:08.864825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 495/1000] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:47:08 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 10:47:08.864825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:08.889626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 506/1001] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:08.931172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:08.991416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcb240-cf18-46ca-ac4d-392e93257cf4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:09.028313 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:09.028928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:09.029204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:09.029468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:09.030126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:09.030447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:09.031797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:09.037048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 496/1002] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:47:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:09.042986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4398f76-0461-435c-8d6e-2b5f28759c82 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:09.146649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4398f76-0461-435c-8d6e-2b5f28759c82 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:09.155516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:09.183826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:09.185078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:09.185358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:09.185627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:09.186269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:09.186564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:09.187922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:09.192890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 507/1003] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 22 10:47:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:09.198210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:09.241681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:09.268842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4437b1-381e-493d-b66d-78065097515d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:09.283819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:09.304182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:09.304817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:09.305101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:09.305654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:09.306322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:09.308215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:09.318135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:09.319538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:09.320385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 497/1004] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 22 10:47:09 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:09.325791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:09.329265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d97eb4-4f9c-4907-9722-56f84efc1d5b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:09.368161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:09.369416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:09.369912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:09.370900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:09.372348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:09.374954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:09.437057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b40ad3f-99e1-4cb6-bee3-5b45b2558ed4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:09.445386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 508/1005] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:47:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:47:09.486866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd6bc1-ec35-4a63-9978-75d375f84e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:09.487782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd6bc1-ec35-4a63-9978-75d375f84e72 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:09.488217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd6bc1-ec35-4a63-9978-75d375f84e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:09.488697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd6bc1-ec35-4a63-9978-75d375f84e72 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:09.489495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd6bc1-ec35-4a63-9978-75d375f84e72 None admin] RBAC: Adding query filter params (name=image) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:09.490004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd6bc1-ec35-4a63-9978-75d375f84e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:09.491707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd6bc1-ec35-4a63-9978-75d375f84e72 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:09.498623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 498/1006] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:09.504133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:09.613836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5e63-f528-4709-82a8-b6284735d88c None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:09.615491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:09.655284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec150664-18b8-419a-9aba-c928fb8ac685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:09.656240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec150664-18b8-419a-9aba-c928fb8ac685 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:09.656458 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec150664-18b8-419a-9aba-c928fb8ac685 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:09.656629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec150664-18b8-419a-9aba-c928fb8ac685 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:09.657264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec150664-18b8-419a-9aba-c928fb8ac685 None admin] RBAC: Adding query filter params (type=image) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:09.657560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec150664-18b8-419a-9aba-c928fb8ac685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:09.658946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec150664-18b8-419a-9aba-c928fb8ac685 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:09.664145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 509/1007] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:09 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:10.792877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146aeff-374c-4b70-8cdb-a5eec7006da7 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:10.793490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146aeff-374c-4b70-8cdb-a5eec7006da7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:10.793490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146aeff-374c-4b70-8cdb-a5eec7006da7 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:10.794890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 499/1008] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:10.799916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3b6b-0b19-41a9-90f2-0c4fe4cfdcf8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:10.800309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3b6b-0b19-41a9-90f2-0c4fe4cfdcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:10.800467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec3b6b-0b19-41a9-90f2-0c4fe4cfdcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:10.819518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b8ec3b6b-0b19-41a9-90f2-0c4fe4cfdcf8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:10.820706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b8ec3b6b-0b19-41a9-90f2-0c4fe4cfdcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:10.894077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 510/1009] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:10 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:11.163337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146aeff-374c-4b70-8cdb-a5eec7006da7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:11.206573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:11.269216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146aeff-374c-4b70-8cdb-a5eec7006da7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:11.303141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b2c6e-d453-4098-8e93-9c0bea750a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:11.303774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b2c6e-d453-4098-8e93-9c0bea750a0d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.304051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b2c6e-d453-4098-8e93-9c0bea750a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.304318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b2c6e-d453-4098-8e93-9c0bea750a0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.304891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b2c6e-d453-4098-8e93-9c0bea750a0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:11.306265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b2c6e-d453-4098-8e93-9c0bea750a0d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-d09b2c6e-d453-4098-8e93-9c0bea750a0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:11.315973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:11.316974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 22 10:47:11.317795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:11.321954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 500/1010] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:11.323412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec3b6b-0b19-41a9-90f2-0c4fe4cfdcf8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:11.405512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c46343-d31b-4e34-b4ac-d0b7eace7fed None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.406218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c46343-d31b-4e34-b4ac-d0b7eace7fed None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.406599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c46343-d31b-4e34-b4ac-d0b7eace7fed None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.408028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 501/1011] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:47:11.413042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb4f1b-0a1b-4889-afc7-3f65e92bdcef None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.413322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb4f1b-0a1b-4889-afc7-3f65e92bdcef None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.413649 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb4f1b-0a1b-4889-afc7-3f65e92bdcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.427667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:11.431725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-cabb4f1b-0a1b-4889-afc7-3f65e92bdcef None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:11.432906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cabb4f1b-0a1b-4889-afc7-3f65e92bdcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:11.434489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec3b6b-0b19-41a9-90f2-0c4fe4cfdcf8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:11.478914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:11.480156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.481536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.482029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.482898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:11.483405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:11.485227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:11.491078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 511/1012] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:47:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:11.497256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:11.516880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 502/1013] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:11.598089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1ba73-a7eb-4007-b9ab-67e17e2de204 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:11.623585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2a29-b999-486f-9a37-0eaaee9c4110 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.623904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2a29-b999-486f-9a37-0eaaee9c4110 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.624131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2a29-b999-486f-9a37-0eaaee9c4110 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.625476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 503/1014] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:47:11.630515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c5ecc-248e-4b9e-a3dd-b5a9959ff4bf None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.630962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c5ecc-248e-4b9e-a3dd-b5a9959ff4bf None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.631383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c5ecc-248e-4b9e-a3dd-b5a9959ff4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.634328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:11.634928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.635204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.635474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.636095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:11.636391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:11.638322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:11.643804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 512/1015] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 22 10:47:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:11.649265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:11.651347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-042c5ecc-248e-4b9e-a3dd-b5a9959ff4bf None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:11.652797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-042c5ecc-248e-4b9e-a3dd-b5a9959ff4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:11.676634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:11.680248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:11.720977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 504/1016] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:11.755277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acfe5f-6dbf-4f8b-8169-8e4512b7d681 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:11.785770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c5ecc-248e-4b9e-a3dd-b5a9959ff4bf None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:11.788706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:11.789319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.789594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.789864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.790550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:11.791887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:11.864802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 513/1017] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:47:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:11.901022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c5ecc-248e-4b9e-a3dd-b5a9959ff4bf None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:11.947127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:11.947745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:11.948030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:11.948300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:11.949927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:11.951438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:11.954283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 505/1018] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:11.958979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:11.993530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:12.024306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:12.055665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:12.081393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51bab9-e744-405b-abac-5a1ad57c1c4b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:12.114930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c7153-c148-451d-905b-8b6edf666703 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:12.125652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1838e71-6bc0-48d0-bd02-58987bf142b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:12.126293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1838e71-6bc0-48d0-bd02-58987bf142b8 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:12.126579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1838e71-6bc0-48d0-bd02-58987bf142b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:12.126858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1838e71-6bc0-48d0-bd02-58987bf142b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:12.127516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1838e71-6bc0-48d0-bd02-58987bf142b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:12.128941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1838e71-6bc0-48d0-bd02-58987bf142b8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:12.150415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad18a-4012-4c87-baa8-0c5dd06bd4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:12.151056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad18a-4012-4c87-baa8-0c5dd06bd4b1 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:12.151325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad18a-4012-4c87-baa8-0c5dd06bd4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:12.151593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad18a-4012-4c87-baa8-0c5dd06bd4b1 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:12.152270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad18a-4012-4c87-baa8-0c5dd06bd4b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:12.154440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad18a-4012-4c87-baa8-0c5dd06bd4b1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:12.194307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 514/1019] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:47:11 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:47:12.210614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 506/1020] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:47:11 2022] POST /identity/v3/services => generated 242 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:47:13.882473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e42da-1f60-4e15-9034-1d64634f7293 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:13.882969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e42da-1f60-4e15-9034-1d64634f7293 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:13.883070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e42da-1f60-4e15-9034-1d64634f7293 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:13.884485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 515/1021] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:13.890139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a40c7-4c0a-4007-9cc0-17cc92c55cab None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:13.890566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a40c7-4c0a-4007-9cc0-17cc92c55cab None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:13.890744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a40c7-4c0a-4007-9cc0-17cc92c55cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:13.909119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6d7a40c7-4c0a-4007-9cc0-17cc92c55cab None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:13.910374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6d7a40c7-4c0a-4007-9cc0-17cc92c55cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:13.986263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 507/1022] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:14.147284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54276827-daed-4863-9f66-3f63a5f45248 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.147586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54276827-daed-4863-9f66-3f63a5f45248 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.147773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54276827-daed-4863-9f66-3f63a5f45248 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.149219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 516/1023] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:14.155032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d63ce-08cf-47c1-9f87-fe65121bf0a8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.155322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d63ce-08cf-47c1-9f87-fe65121bf0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.155600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d63ce-08cf-47c1-9f87-fe65121bf0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.173610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-063d63ce-08cf-47c1-9f87-fe65121bf0a8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:14.174809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-063d63ce-08cf-47c1-9f87-fe65121bf0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:14.182498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db93000-612f-4e5e-ab00-e4bc4cb2153e None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.182780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db93000-612f-4e5e-ab00-e4bc4cb2153e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.183049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db93000-612f-4e5e-ab00-e4bc4cb2153e None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.183173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:14.184530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 517/1024] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:47:14.189405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa79dfb-7413-4d33-96be-03d81ea2b9f0 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.189687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa79dfb-7413-4d33-96be-03d81ea2b9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.189962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa79dfb-7413-4d33-96be-03d81ea2b9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.207781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2aa79dfb-7413-4d33-96be-03d81ea2b9f0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:14.212758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2aa79dfb-7413-4d33-96be-03d81ea2b9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:14.245917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 508/1025] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:14.256315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d63ce-08cf-47c1-9f87-fe65121bf0a8 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.282267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 518/1026] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:14.366567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d63ce-08cf-47c1-9f87-fe65121bf0a8 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.411269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:14.411836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.412118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.412393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.423266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:14.427952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:14.428894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:14.429744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:14.434215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 509/1027] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 22 10:47:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:14.435336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa79dfb-7413-4d33-96be-03d81ea2b9f0 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.498142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:14.521821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.525297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:14.536883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa79dfb-7413-4d33-96be-03d81ea2b9f0 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.569907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:14.570595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.570869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.571140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.571792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:14.572087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:14.573461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:14.581633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 519/1028] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 22 10:47:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:14.582445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.624234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39666136-dd72-42a5-9c9e-9a7761d5de4e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.659400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:14.660039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.660317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.660587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] PATH_INFO: `/v3/services/image` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.661164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:14.662626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:14.672072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:14.673256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:14.674155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 510/1029] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 22 10:47:14 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:14.675115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.688010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7f6d0-160c-4ed5-a6f8-a49dea7d24c7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.711942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:14.723472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:14.724113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.724388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.724661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.725786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:14.727411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:14.728971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 520/1030] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:14.729813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.751494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:14.786751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a65a1e-ddca-48b1-93ee-db5caf8e0955 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.821396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:14.822034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.822368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.822625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.833036 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:14.834506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:14.837616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f2092-2952-4448-b53b-0a9df2344438 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:14.848179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:14.849317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:14.850705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:14.850705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:14.850705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:14.850705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 511/1031] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:14.850705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.875982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:14.876593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.876873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.877145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.877775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] RBAC: Adding query filter params (name=image) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:14.878077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:14.879471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:14.884502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 521/1032] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:14.885370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:14.939627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:14.957100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3de2b-6199-4378-b6fc-895356507246 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.974483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:14.983836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33554de3-5952-41b4-8076-94edb0357a9a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:14.990691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:14.991301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:14.991579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:14.991850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:14.992488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:14.994177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.018426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.019064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.019337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.019602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.020234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:15.020531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.021872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.034183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 522/1033] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:15.035019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:15.089260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 512/1034] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:47:14 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:15.090168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:15.155605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c92153-094b-427d-8808-d7d349bddba7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:15.175855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:15.198373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.198913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.199972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.200359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.201124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] RBAC: Adding query filter params (type=image) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:15.201462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.202902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.208310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 523/1035] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:15.216551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:15.220176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4b390-b8f7-46dd-8312-570edd1eec74 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:15.257124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.257773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.258064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.258449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.267491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.269026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:47:15.283170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:15.284158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:15.285112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 513/1036] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:47:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:15.290103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:15.330038 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:15.342203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa61c1-6694-4599-be36-8a27e4fe0593 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:15.381950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.384208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.384536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.384977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.385784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] RBAC: Adding query filter params (interface=public, service_id=ef54ccd32cd54679b1468382144daf45, region_id=RegionOne) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:15.386104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.388574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.395583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 524/1037] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:47:15 2022] GET /identity/v3/endpoints?service_id=ef54ccd32cd54679b1468382144daf45&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:15.400869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:15.412392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf56337-0fee-454f-8eca-ec28147b51d1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:15.449700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:15.450669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.451656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.452149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.452562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.453356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:15.453662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.455145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.473774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 514/1038] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:47:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:15.479119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:15.514595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609a346-7316-4676-af36-33ee683e8d3e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:15.558123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.559243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.559676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.560090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.560844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.562265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.568848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 525/1039] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:47:15 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 10:47:15.598627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:15.609703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f663-a1de-41a2-be14-59bfa57febb0 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:15.648881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.649512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.649791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.650057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.722410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.724082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:15.816010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:15.826685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 515/1040] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:15 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 10:47:15.832247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:15.937628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186b396-d431-45c2-9644-bd464d6a909e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:15.947615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:15.972582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:15.973134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:15.973411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:15.973683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:15.984615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:15.986057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:16.001148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:16.002101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:16.003066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:16.003066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:16.003066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:16.003066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 526/1041] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:16.011506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:16.114336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaab400-d205-4b5d-a0f5-a379100f9bee None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:16.148386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:16.149000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:16.149280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:16.149552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:16.150187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:16.150515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:16.151891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:16.164289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 516/1042] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:16.170586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:16.271324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d3f79-f8f4-4fa8-b7d4-73c34896a49d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:16.304687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:16.305305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:16.305581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:16.305849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:16.312468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:16.313884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:47:16.325130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:16.326018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:16.327203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:16.328046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 527/1043] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:16.332182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:16.389124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:16.436710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd4f3a-6489-4246-bf23-863cdabeb0a8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:16.471006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71cbef-9586-4aa1-9b02-dc2a7d06cf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:16.471611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71cbef-9586-4aa1-9b02-dc2a7d06cf71 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:16.471893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71cbef-9586-4aa1-9b02-dc2a7d06cf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:16.472162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71cbef-9586-4aa1-9b02-dc2a7d06cf71 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:16.472837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71cbef-9586-4aa1-9b02-dc2a7d06cf71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:16.473139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71cbef-9586-4aa1-9b02-dc2a7d06cf71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:16.474583 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71cbef-9586-4aa1-9b02-dc2a7d06cf71 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:16.487426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 517/1044] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:16.493232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:16.517690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:16.597317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87901d-3863-4b4b-84eb-22d13e9f8c1d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:16.630881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44b870-7623-487f-bc67-7da7c3b50ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:16.631487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b870-7623-487f-bc67-7da7c3b50ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:16.631761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b870-7623-487f-bc67-7da7c3b50ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:16.632030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44b870-7623-487f-bc67-7da7c3b50ac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:16.632744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44b870-7623-487f-bc67-7da7c3b50ac8 None admin] RBAC: Adding query filter params (role.id=aa67480f49d64305ac7e9d005617826a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7f5fe23bc8294bc3861e0db9dad2045e) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:16.633047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44b870-7623-487f-bc67-7da7c3b50ac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:16.634767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44b870-7623-487f-bc67-7da7c3b50ac8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:16.646268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 528/1045] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:16 2022] GET /identity/v3/role_assignments?user.id=7f5fe23bc8294bc3861e0db9dad2045e&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=aa67480f49d64305ac7e9d005617826a => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:17.075601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cab9f-4418-476f-a6bd-bc173d7a4324 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:17.076104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cab9f-4418-476f-a6bd-bc173d7a4324 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:17.076179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cab9f-4418-476f-a6bd-bc173d7a4324 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:17.077624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 518/1046] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:17.082680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5e0c-d551-47dd-bcf6-dd5911b935bc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:17.083069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5e0c-d551-47dd-bcf6-dd5911b935bc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:17.083242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa5e0c-d551-47dd-bcf6-dd5911b935bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:17.101874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-26fa5e0c-d551-47dd-bcf6-dd5911b935bc None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:17.103092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-26fa5e0c-d551-47dd-bcf6-dd5911b935bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:17.171246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 529/1047] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:17.512511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcff89f-fe1f-49c0-9566-c75931ed44fe None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:17.512790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcff89f-fe1f-49c0-9566-c75931ed44fe None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:17.513045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cab9f-4418-476f-a6bd-bc173d7a4324 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:17.513152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcff89f-fe1f-49c0-9566-c75931ed44fe None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:17.514610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 530/1048] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:17.520033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9587-20ce-4c00-ac4b-802bff22cf2e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:17.520311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9587-20ce-4c00-ac4b-802bff22cf2e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:17.520583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d9587-20ce-4c00-ac4b-802bff22cf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:17.609834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-da6d9587-20ce-4c00-ac4b-802bff22cf2e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:17.611176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-da6d9587-20ce-4c00-ac4b-802bff22cf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:17.623740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:17.632494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cab9f-4418-476f-a6bd-bc173d7a4324 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:17.653129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:17.678432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:17.679112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:17.679390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:17.679661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:17.694642 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:17.699542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:17.700950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:17.701801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 519/1049] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:47:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:17.705620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 531/1050] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 22 10:47:17.707522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:17.841697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee58e6a-321b-4d7d-b9af-1a823122b4d9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:17.893227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:17.893227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:17.893227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:17.893616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:17.894113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:17.894493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:17.895860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:17.911103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 520/1051] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:47:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:17.913400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d9587-20ce-4c00-ac4b-802bff22cf2e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:17.979145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:18.018923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:18.046193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d9587-20ce-4c00-ac4b-802bff22cf2e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.097188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.097818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.098032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.098315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] PATH_INFO: `/v3/users/61759dde89ca4de5a1454a5cd974fa22` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.101488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a0416-0fb9-44a1-a44d-fe48e10dba36 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.117817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=61759dde89ca4de5a1454a5cd974fa22)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.120889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.127726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 532/1052] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:17 2022] GET /identity/v3/users/61759dde89ca4de5a1454a5cd974fa22 => generated 285 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:18.129837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:18.133456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:18.149589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.150365 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.150695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.150965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.151547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.157463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:18.168451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:18.169493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:18.170363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 521/1053] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 22 10:47:17 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:18.175981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:18.266977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4b982-ee61-464f-8905-6e6a6e7c8cd7 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.290298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:18.294753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ff805-425b-4c30-9d68-a3cc0f54eba7 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.302416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.303383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.303383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.303383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.333185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.333799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.334075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.334373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.334999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] RBAC: Adding query filter params (name=image) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:18.335298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.336653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.346144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 522/1054] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:18.347175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:18.378552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.379904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.381728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:18.454532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128719c2-c8c0-47ab-9954-7cf2b36d68dd None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.489760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 533/1055] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:18.492561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.493350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.494041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.494480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.495290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:18.496359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.496840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:18.498121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.506273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 523/1056] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:18.509108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:18.540392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:18.627128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b80f4d-d2e0-421f-b871-02ddd3b25ca1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.627358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a1f85-10de-42d7-ac36-2ad3c0b9534a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.628786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:18.662218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2100f83-f222-4532-b422-41a843203bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.662876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2100f83-f222-4532-b422-41a843203bae None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.663154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2100f83-f222-4532-b422-41a843203bae None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.663427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2100f83-f222-4532-b422-41a843203bae None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.664540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2100f83-f222-4532-b422-41a843203bae None admin] RBAC: Authorizing `identity:get_project(project_id=870dfee6c2a84684bd660d8e17688b4c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.665449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5bc39-6042-400a-aac5-6040f10596e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.665964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2100f83-f222-4532-b422-41a843203bae None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.666063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bc39-6042-400a-aac5-6040f10596e0 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.666366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bc39-6042-400a-aac5-6040f10596e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.666640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bc39-6042-400a-aac5-6040f10596e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.667296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5bc39-6042-400a-aac5-6040f10596e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.668361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 534/1057] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:18 2022] GET /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:18.669326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5bc39-6042-400a-aac5-6040f10596e0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.674453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2100f83-f222-4532-b422-41a843203bae None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:18.745734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 524/1058] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:47:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 10:47:18.749048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1158b9-4da9-4797-b98a-eeea3ba60cda None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.749048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1158b9-4da9-4797-b98a-eeea3ba60cda None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.749048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1158b9-4da9-4797-b98a-eeea3ba60cda None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.749048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 525/1059] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:18.754371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf8a6-5af4-4f41-8476-a24536934085 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.754770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf8a6-5af4-4f41-8476-a24536934085 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.755045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4bf8a6-5af4-4f41-8476-a24536934085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.775635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-eb4bf8a6-5af4-4f41-8476-a24536934085 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:18.776577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-eb4bf8a6-5af4-4f41-8476-a24536934085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:18.779658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2100f83-f222-4532-b422-41a843203bae None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:18.827045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:18.827675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:18.827955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:18.828230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:18.828972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=870dfee6c2a84684bd660d8e17688b4c, user.id=61759dde89ca4de5a1454a5cd974fa22) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:18.829280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:18.830788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:18.836738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 535/1060] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:18 2022] GET /identity/v3/role_assignments?user.id=61759dde89ca4de5a1454a5cd974fa22&scope.project.id=870dfee6c2a84684bd660d8e17688b4c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:18.875538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 526/1061] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:19.142652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:19.175342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:19.247350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947b4d0-0359-4432-8db2-0723ff330c38 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:19.283195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:19.283830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:19.284109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:19.284385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:19.294829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:19.299968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:19.300889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:19.301710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:19.305974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 536/1062] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 22 10:47:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:19.307353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bf8a6-5af4-4f41-8476-a24536934085 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:19.321541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:19.417249 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4bf8a6-5af4-4f41-8476-a24536934085 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:19.451622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:19.452229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:19.452505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:19.452774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:19.453434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:19.453733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:19.455135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:19.463419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 527/1063] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 22 10:47:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:19.469591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:19.562902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:19.572567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e72e49-ba1f-42bb-869b-936dcb36f8eb None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:19.606637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:19.607259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:19.607535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:19.607803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:19.617449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:19.619183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:19.633711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:19.634682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:19.635583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:19.635583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:19.635583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:19.640221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 537/1064] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:19.641558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:19.716097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:19.741069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8ab56-a2ca-4579-b01d-7c1ae813861f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:19.776130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:19.776804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:19.777080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:19.777351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:19.777991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:19.778320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:19.779675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:19.791855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 528/1065] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:19.797584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:19.910153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b0a0f-4e0e-49b9-87ba-7d58387409ae None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:19.943721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:19.944370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:19.944646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:19.944912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:19.952950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:19.954387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:47:19.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:19.971890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:19.972867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 538/1066] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:47:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:19.975167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:20.079166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dd9dc-2d2b-4dab-a1d2-9cd95fecce76 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:20.114738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:20.115486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.115870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.116244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.117126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:20.117531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:20.119349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:20.121799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:20.139270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 529/1067] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:47:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:20.144892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:20.147919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:20.249116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d39f-09cc-4f59-a9c0-728888048392 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:20.283880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:20.284511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.284793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.285068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.296397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:20.297848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:20.312899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:20.313880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:20.314791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:20.314791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:20.314791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:20.318551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 539/1068] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:20.319909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:20.420711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d18ce3-b5fe-4f54-962c-9c29426e2aef None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:20.455783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:20.456395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.456673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.456948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.457588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:20.457887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:20.459268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:20.475488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 530/1069] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:20.477123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:20.534115 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:20.579531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66f434-0fae-4b96-b890-fbc10eb493f5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:20.613500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef7a0f-43d8-4b0e-944a-4c13ef893ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:20.614216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef7a0f-43d8-4b0e-944a-4c13ef893ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.614583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef7a0f-43d8-4b0e-944a-4c13ef893ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.614920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef7a0f-43d8-4b0e-944a-4c13ef893ab0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.622533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef7a0f-43d8-4b0e-944a-4c13ef893ab0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:20.624025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef7a0f-43d8-4b0e-944a-4c13ef893ab0 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-6bef7a0f-43d8-4b0e-944a-4c13ef893ab0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:47:20.634891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:20.635933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:20.637090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:20.642390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 540/1070] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:20.643404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:20.699737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:20.719204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c1e3-0fd6-4ac3-bfdb-5eb48005f583 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.719682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c1e3-0fd6-4ac3-bfdb-5eb48005f583 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.719956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4992c1e3-0fd6-4ac3-bfdb-5eb48005f583 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.721578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 541/1071] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:47:20.726538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38652c61-1d5a-4709-8cd2-4a5089750bd7 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.726825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38652c61-1d5a-4709-8cd2-4a5089750bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.727105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38652c61-1d5a-4709-8cd2-4a5089750bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.745363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-38652c61-1d5a-4709-8cd2-4a5089750bd7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:20.746663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-38652c61-1d5a-4709-8cd2-4a5089750bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:20.748823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218366-1e3a-49eb-bc4d-c1acf1b0f763 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:20.782786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e691c7d-8cd0-47f9-8ff2-02e27735bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:20.783401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e691c7d-8cd0-47f9-8ff2-02e27735bfae None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.783683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e691c7d-8cd0-47f9-8ff2-02e27735bfae None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.783955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e691c7d-8cd0-47f9-8ff2-02e27735bfae None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.784697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e691c7d-8cd0-47f9-8ff2-02e27735bfae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:20.784950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e691c7d-8cd0-47f9-8ff2-02e27735bfae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:20.786425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e691c7d-8cd0-47f9-8ff2-02e27735bfae None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:20.798310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 531/1072] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:20.799491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed0ad-58b8-48b4-9735-d6df532097c5 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.799768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed0ad-58b8-48b4-9735-d6df532097c5 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.800046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeed0ad-58b8-48b4-9735-d6df532097c5 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.801379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 532/1073] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:20.804401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeed0ad-58b8-48b4-9735-d6df532097c5 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:20.813255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 542/1074] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:20.814528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab4874-a19e-4bde-a16a-de4fcba2445c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.814812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab4874-a19e-4bde-a16a-de4fcba2445c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.815088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab4874-a19e-4bde-a16a-de4fcba2445c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.834593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-11ab4874-a19e-4bde-a16a-de4fcba2445c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:20.835782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-11ab4874-a19e-4bde-a16a-de4fcba2445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:20.883566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:20.912002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 543/1075] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:20.923973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeed0ad-58b8-48b4-9735-d6df532097c5 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:20.937803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:20.959150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:20.959820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:20.960100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:20.960372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7f5fe23bc8294bc3861e0db9dad2045e/roles/aa67480f49d64305ac7e9d005617826a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:20.962592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=7f5fe23bc8294bc3861e0db9dad2045e, role_id=aa67480f49d64305ac7e9d005617826a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:20.964025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:21.031883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 533/1076] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:47:20 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/7f5fe23bc8294bc3861e0db9dad2045e/roles/aa67480f49d64305ac7e9d005617826a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 10:47:21.083891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab4874-a19e-4bde-a16a-de4fcba2445c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.189353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab4874-a19e-4bde-a16a-de4fcba2445c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:21.204817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.226101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:21.226739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:21.227021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:21.227292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:21.271828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:21.300943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:21.302419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:21.316329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf529ab-f60e-45b7-8e8c-c59d16fc09a4 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:21.355583 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:21.356194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:21.356428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:21.356666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:21.368039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:21.373631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:21.374594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:21.375558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 534/1077] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:21.380692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.414258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 544/1078] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:21.418249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.418592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:21.494106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52009ae9-a1a9-4f82-9d6c-02e77c01d3c0 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:21.502143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:21.532785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:21.533393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:21.533667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:21.533936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:21.534620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:21.534923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:21.536284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:21.537029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7110c4-69e4-4e4d-b8c0-e55a3498a986 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:21.546813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 535/1079] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:21.552780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.582213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:21.582939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:21.583316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:21.583491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:21.593871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:21.595441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:21.611221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:21.612145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:21.613015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:21.613015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:21.613015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:21.617276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 545/1080] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:21.617927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.657652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:21.666903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17446ac4-ee6c-4be8-9275-9ea86ddebfc2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:21.704678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:21.705309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:21.705592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:21.705863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] PATH_INFO: `/v3/users/61759dde89ca4de5a1454a5cd974fa22` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:21.707015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] RBAC: Authorizing `identity:get_user(user_id=61759dde89ca4de5a1454a5cd974fa22)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:21.708438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:21.710220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 536/1081] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:21 2022] GET /identity/v3/users/61759dde89ca4de5a1454a5cd974fa22 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:21.715849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.727562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f6d5c-128c-4b45-be74-fb2a645964e5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:21.755641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:21.765706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:21.766331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:21.768082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:21.768082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:21.768082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:21.768082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:21.769280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:21.782630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 546/1082] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:21.819832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b18b2-6b71-4bfa-84bd-07fc84ec12d2 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:21.869255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555abd8-7f9b-4b81-9c57-dc9baaafebc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:21.869998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555abd8-7f9b-4b81-9c57-dc9baaafebc7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:21.871008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555abd8-7f9b-4b81-9c57-dc9baaafebc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:21.871383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555abd8-7f9b-4b81-9c57-dc9baaafebc7 None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:21.872705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555abd8-7f9b-4b81-9c57-dc9baaafebc7 None admin] RBAC: Authorizing `identity:get_project(project_id=870dfee6c2a84684bd660d8e17688b4c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:21.874295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555abd8-7f9b-4b81-9c57-dc9baaafebc7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:21.876276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 537/1083] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:21 2022] GET /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:21.882277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:21.991023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9a57c-3e88-44b4-a3d0-b6002ced6b92 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:22.031203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d92094-894d-4495-9a67-204988761a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:22.031875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92094-894d-4495-9a67-204988761a40 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:22.032157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92094-894d-4495-9a67-204988761a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:22.032430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92094-894d-4495-9a67-204988761a40 None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c/users/61759dde89ca4de5a1454a5cd974fa22/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:22.034401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d92094-894d-4495-9a67-204988761a40 None admin] RBAC: Authorizing `identity:create_grant(project_id=870dfee6c2a84684bd660d8e17688b4c, user_id=61759dde89ca4de5a1454a5cd974fa22, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:22.035836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d92094-894d-4495-9a67-204988761a40 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:22.106318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 547/1084] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:47:21 2022] PUT /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c/users/61759dde89ca4de5a1454a5cd974fa22/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:47:22.954026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34833266-bdfc-4d60-b83f-99a7e7fe5ee2 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:22.954620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34833266-bdfc-4d60-b83f-99a7e7fe5ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:22.954620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34833266-bdfc-4d60-b83f-99a7e7fe5ee2 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:22.955962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 538/1085] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:22.961222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded6c8-62ac-4610-9ef6-90a8a15ad129 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:22.961624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded6c8-62ac-4610-9ef6-90a8a15ad129 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:22.961771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded6c8-62ac-4610-9ef6-90a8a15ad129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:22.979547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-1fded6c8-62ac-4610-9ef6-90a8a15ad129 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:22.981345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1fded6c8-62ac-4610-9ef6-90a8a15ad129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:23.022448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:23.046220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 548/1086] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:23.319601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34833266-bdfc-4d60-b83f-99a7e7fe5ee2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:23.349915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:23.420038 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34833266-bdfc-4d60-b83f-99a7e7fe5ee2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:23.455490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:23.456137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:23.456419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:23.456696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:23.467896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:23.472661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:23.473686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:23.474966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 539/1087] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 22 10:47:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:23.479831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fded6c8-62ac-4610-9ef6-90a8a15ad129 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:23.580014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fded6c8-62ac-4610-9ef6-90a8a15ad129 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:23.613566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:23.614198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:23.614507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:23.614783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:23.615447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:23.615748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:23.617105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:23.625958 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 549/1088] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 22 10:47:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:23.631592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:23.721851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:23.730820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b73a24-7818-4774-8033-a17bca6d87d7 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:23.765571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:23.766192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:23.766513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:23.766781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:23.777312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:23.779157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:23.794666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:23.795734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:23.796956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:23.796956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:23.796956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:23.796956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 540/1089] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:23.801652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:23.843754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:23.904003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414dae2-a835-4f9b-89ab-c9ce1f3bc748 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:23.937033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c71b5d-6e7d-41c3-949e-06f0b4a662e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:23.937643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c71b5d-6e7d-41c3-949e-06f0b4a662e8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:23.937925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c71b5d-6e7d-41c3-949e-06f0b4a662e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:23.938199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c71b5d-6e7d-41c3-949e-06f0b4a662e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:23.938876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c71b5d-6e7d-41c3-949e-06f0b4a662e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:23.939181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c71b5d-6e7d-41c3-949e-06f0b4a662e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:23.940555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c71b5d-6e7d-41c3-949e-06f0b4a662e8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:23.952786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 550/1090] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:23.958373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:23.991878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfbb22-09dc-4177-bb83-2e1885bbe7ad None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:23.992364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfbb22-09dc-4177-bb83-2e1885bbe7ad None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:23.992550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfbb22-09dc-4177-bb83-2e1885bbe7ad None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:23.994160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 551/1091] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:47:23.999271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ac2b8-c7fe-4683-951f-d48597f7c055 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:23.999552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ac2b8-c7fe-4683-951f-d48597f7c055 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:23.999824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ac2b8-c7fe-4683-951f-d48597f7c055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.017668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-107ac2b8-c7fe-4683-951f-d48597f7c055 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:24.018893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-107ac2b8-c7fe-4683-951f-d48597f7c055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:24.060688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296563-2a07-4da3-8a4d-e861c3f98748 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.085920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 552/1092] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:24.094730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.095305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.095581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.095849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.103083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:24.104503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:47:24.115644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:24.122329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:24.123573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 541/1093] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 22 10:47:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:24.124298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ac2b8-c7fe-4683-951f-d48597f7c055 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.151262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.220898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ac2b8-c7fe-4683-951f-d48597f7c055 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.253725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.254346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.254631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.254933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.255654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:24.255957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:24.257361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:24.274595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 553/1094] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:24.280348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.283307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.354080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.379205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f254b9-0a07-466e-a989-8ac10e245d99 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.412212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.412909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.413187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.413456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.438999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.454057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3bf34-c6b6-425c-a05b-7bf6cdfdca0d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.481409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:24.482894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:24.484686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.490977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.491944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.495695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.495695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.506566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:24.512240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:24.513333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:24.514585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 554/1095] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:24.519327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.587063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 542/1096] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 10:47:24.593312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.616113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf695-4542-4abd-a5cc-115d38d4ce48 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.649963 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.650586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.650861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.651138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.652433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:24.652730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:24.654089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:24.663037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 555/1097] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:24.668831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.678773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.690252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcacc0-9898-4ccb-bda9-f4e4cfe5db0a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.691788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.724671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.725303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.725589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.725834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.736555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:24.738046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:24.753110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:24.754196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:47:24.755117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:24.755117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:47:24.755117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:24.755117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 543/1098] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:24.759765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.767610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8836f-603e-4041-9bae-6e157da71569 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.809681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.810329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.810603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.810881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] PATH_INFO: `/v3/users/61759dde89ca4de5a1454a5cd974fa22` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.811991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] RBAC: Authorizing `identity:get_user(user_id=61759dde89ca4de5a1454a5cd974fa22)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:24.813402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:24.815003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 556/1099] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/users/61759dde89ca4de5a1454a5cd974fa22 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:24.820727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.872569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf54f35-0aae-4c3d-ade5-9cd2a365414b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.899298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.908360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.913810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.913810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.913810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:24.913810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:24.913810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:24.913810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:24.922765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edae35-0964-4043-8b2c-b1e9ce53ffde None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:24.925050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 544/1100] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:24.930710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:24.946077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:24.956963 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:24.957566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:24.957839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:24.959089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:25.027339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:25.028771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:25.030808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4631e5-a89b-45d4-8330-ab414217bd8e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:25.065165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:25.065797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:25.066078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:25.066383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:25.073054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:25.075195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:47:25.086164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:25.087246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:25.088329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 545/1101] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:47:25.093013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:25.103832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:25.127525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 557/1102] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 22 10:47:25.133538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:25.178454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:25.205442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52790389-b19b-4f03-86c6-7463e3249d85 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:25.239176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:25.239833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:25.240112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:25.240387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:25.241061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:25.241361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:25.242812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:25.246821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf310e-24ac-4ea6-9890-d4c30ab6a530 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:25.255974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 546/1103] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:25.261774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:25.280509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:25.281144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:25.281424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:25.281700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:25.282827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] RBAC: Authorizing `identity:get_project(project_id=870dfee6c2a84684bd660d8e17688b4c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:25.284246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:25.285877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 558/1104] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:25 2022] GET /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:25.291761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:25.336352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:25.361665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab670c-ea5c-47c9-8ab5-03ddf5941066 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:25.389419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121cfa6-93fa-4337-83f9-ddc18b600a4a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:25.395161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc5fed-943d-4909-a6ae-6dde2f1ca1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:25.395767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5fed-943d-4909-a6ae-6dde2f1ca1dd None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:25.396046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5fed-943d-4909-a6ae-6dde2f1ca1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:25.396321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5fed-943d-4909-a6ae-6dde2f1ca1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:25.397050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc5fed-943d-4909-a6ae-6dde2f1ca1dd None admin] RBAC: Adding query filter params (role.id=aa67480f49d64305ac7e9d005617826a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=7f5fe23bc8294bc3861e0db9dad2045e) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:25.397356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc5fed-943d-4909-a6ae-6dde2f1ca1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:25.398834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc5fed-943d-4909-a6ae-6dde2f1ca1dd None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:25.400545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:25.405511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 547/1105] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:25 2022] GET /identity/v3/role_assignments?user.id=7f5fe23bc8294bc3861e0db9dad2045e&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=aa67480f49d64305ac7e9d005617826a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:25.424130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ea86b-a49f-4ccc-8500-be75f25182da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:25.424800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ea86b-a49f-4ccc-8500-be75f25182da None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:25.425082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ea86b-a49f-4ccc-8500-be75f25182da None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:25.425360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ea86b-a49f-4ccc-8500-be75f25182da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:25.426105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ea86b-a49f-4ccc-8500-be75f25182da None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=870dfee6c2a84684bd660d8e17688b4c, user.id=61759dde89ca4de5a1454a5cd974fa22) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:25.426448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ea86b-a49f-4ccc-8500-be75f25182da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:25.427921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ea86b-a49f-4ccc-8500-be75f25182da None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:25.434871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 559/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:25 2022] GET /identity/v3/role_assignments?user.id=61759dde89ca4de5a1454a5cd974fa22&scope.project.id=870dfee6c2a84684bd660d8e17688b4c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:27.388704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7aba-2706-488b-b6ed-b63a7e17f64c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:27.389253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7aba-2706-488b-b6ed-b63a7e17f64c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:27.389253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7aba-2706-488b-b6ed-b63a7e17f64c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:27.390610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 548/1107] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:27.395682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4cb1-a463-4275-bdb8-73c2d7b9f5d7 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:27.396117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4cb1-a463-4275-bdb8-73c2d7b9f5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:27.396213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4cb1-a463-4275-bdb8-73c2d7b9f5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:27.414942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-997c4cb1-a463-4275-bdb8-73c2d7b9f5d7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:27.416121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-997c4cb1-a463-4275-bdb8-73c2d7b9f5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:27.481521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 560/1108] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:27.755016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a7aba-2706-488b-b6ed-b63a7e17f64c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:27.853663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a7aba-2706-488b-b6ed-b63a7e17f64c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:27.891405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8f449-4934-4a0c-8649-db6e9c373b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:27.892033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8f449-4934-4a0c-8649-db6e9c373b8f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:27.892316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8f449-4934-4a0c-8649-db6e9c373b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:27.892586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8f449-4934-4a0c-8649-db6e9c373b8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:27.893698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8f449-4934-4a0c-8649-db6e9c373b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:27.895152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8f449-4934-4a0c-8649-db6e9c373b8f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:27.896713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 549/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:27.903108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c4cb1-a463-4275-bdb8-73c2d7b9f5d7 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:27.941370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:27.998658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c4cb1-a463-4275-bdb8-73c2d7b9f5d7 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:28.031420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60f78f-7c0d-4481-aae5-09ed498975cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:28.031957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60f78f-7c0d-4481-aae5-09ed498975cf None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:28.032235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60f78f-7c0d-4481-aae5-09ed498975cf None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:28.032507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60f78f-7c0d-4481-aae5-09ed498975cf None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:28.033142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60f78f-7c0d-4481-aae5-09ed498975cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:28.034579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60f78f-7c0d-4481-aae5-09ed498975cf None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:28.132901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 561/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:47:27 2022] POST /identity/v3/users => generated 286 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:30.056176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793d639-7ec7-421f-b07f-96df54840735 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:30.056723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793d639-7ec7-421f-b07f-96df54840735 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:30.056723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793d639-7ec7-421f-b07f-96df54840735 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:30.058698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 550/1111] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:30.063915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41483d-b9b2-4b16-8af6-87b9ce9a590f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:30.064307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41483d-b9b2-4b16-8af6-87b9ce9a590f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:30.064465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41483d-b9b2-4b16-8af6-87b9ce9a590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:30.083172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6c41483d-b9b2-4b16-8af6-87b9ce9a590f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:30.084368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6c41483d-b9b2-4b16-8af6-87b9ce9a590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:30.150829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 562/1112] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:30.417266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793d639-7ec7-421f-b07f-96df54840735 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:30.485037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:30.529003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793d639-7ec7-421f-b07f-96df54840735 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:30.561981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:30.562639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:30.562919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:30.563188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] PATH_INFO: `/v3/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:30.564559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] RBAC: Authorizing `identity:get_role(role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:30.565918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:30.567266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 551/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:30 2022] GET /identity/v3/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:30.573774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41483d-b9b2-4b16-8af6-87b9ce9a590f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:30.597932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:30.669931 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41483d-b9b2-4b16-8af6-87b9ce9a590f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:30.702247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:30.702874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:30.703154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:30.703428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] PATH_INFO: `/v3/users/dfbd17246ef94b98be996ca2ffeb36d1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:30.716098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] RBAC: Authorizing `identity:get_user(user_id=dfbd17246ef94b98be996ca2ffeb36d1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:30.717527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:30.719139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 563/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:30 2022] GET /identity/v3/users/dfbd17246ef94b98be996ca2ffeb36d1 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:30.724806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:30.821412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c70d730-5a93-4f22-a3a1-42389cc457e1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:30.856719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:30.857336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:30.857795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:30.857892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:30.860538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:30.924706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:30.926077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:31.020781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 552/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:31.026826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:31.104494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:31.122957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0eeff-3867-4d3b-aba2-d7ca2bf532df None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:31.156468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5b426-3e70-4a44-939c-c7b9436dc5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:31.157083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5b426-3e70-4a44-939c-c7b9436dc5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:31.157357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5b426-3e70-4a44-939c-c7b9436dc5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:31.157623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5b426-3e70-4a44-939c-c7b9436dc5a4 None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:31.158740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5b426-3e70-4a44-939c-c7b9436dc5a4 None admin] RBAC: Authorizing `identity:get_project(project_id=870dfee6c2a84684bd660d8e17688b4c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:31.160141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5b426-3e70-4a44-939c-c7b9436dc5a4 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:31.161772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 564/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:31 2022] GET /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:31.168699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:31.201164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:31.264262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8dfd9-54d7-4cdb-a3c8-53e3612bbb43 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:31.296439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65aec0-d303-47f8-8777-6c9b59b32061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:31.297029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aec0-d303-47f8-8777-6c9b59b32061 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:31.297307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aec0-d303-47f8-8777-6c9b59b32061 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:31.297579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aec0-d303-47f8-8777-6c9b59b32061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:31.298313 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aec0-d303-47f8-8777-6c9b59b32061 None admin] RBAC: Adding query filter params (role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=870dfee6c2a84684bd660d8e17688b4c, user.id=dfbd17246ef94b98be996ca2ffeb36d1) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:31.298618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aec0-d303-47f8-8777-6c9b59b32061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:31.300029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aec0-d303-47f8-8777-6c9b59b32061 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:31.305152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 553/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:31 2022] GET /identity/v3/role_assignments?user.id=dfbd17246ef94b98be996ca2ffeb36d1&scope.project.id=870dfee6c2a84684bd660d8e17688b4c&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:33.149293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc3c8-221d-4ad9-bd65-ec7d9981cee1 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:33.149792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc3c8-221d-4ad9-bd65-ec7d9981cee1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:33.149792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc3c8-221d-4ad9-bd65-ec7d9981cee1 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:33.151656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 565/1118] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:33.156830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e271c7-0eda-4a35-b349-e3365aa605b8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:33.157260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e271c7-0eda-4a35-b349-e3365aa605b8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:33.157352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e271c7-0eda-4a35-b349-e3365aa605b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:33.175395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-45e271c7-0eda-4a35-b349-e3365aa605b8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:33.176557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-45e271c7-0eda-4a35-b349-e3365aa605b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:33.240072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 554/1119] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:33.507604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdc3c8-221d-4ad9-bd65-ec7d9981cee1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:33.605374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdc3c8-221d-4ad9-bd65-ec7d9981cee1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:33.641392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:33.642052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:33.642358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:33.642630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] PATH_INFO: `/v3/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:33.644002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] RBAC: Authorizing `identity:get_role(role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:33.645360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:33.646697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 566/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:33 2022] GET /identity/v3/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:33.652529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e271c7-0eda-4a35-b349-e3365aa605b8 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:33.679341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:33.749135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e271c7-0eda-4a35-b349-e3365aa605b8 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:33.781231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:33.781792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:33.782069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:33.782369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] PATH_INFO: `/v3/users/dfbd17246ef94b98be996ca2ffeb36d1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:33.783457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] RBAC: Authorizing `identity:get_user(user_id=dfbd17246ef94b98be996ca2ffeb36d1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:33.784873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:33.786454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 555/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:33 2022] GET /identity/v3/users/dfbd17246ef94b98be996ca2ffeb36d1 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:33.792320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:33.811020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:33.889878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f746b01-7198-4354-a26d-165aca4607ec None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:33.921825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb8716-de76-474f-9ad6-06313e13c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:33.922468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb8716-de76-474f-9ad6-06313e13c766 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:33.922749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb8716-de76-474f-9ad6-06313e13c766 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:33.923022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb8716-de76-474f-9ad6-06313e13c766 None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:33.924129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb8716-de76-474f-9ad6-06313e13c766 None admin] RBAC: Authorizing `identity:get_project(project_id=870dfee6c2a84684bd660d8e17688b4c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:33.925537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb8716-de76-474f-9ad6-06313e13c766 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:33.927171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 567/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:33 2022] GET /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:33.933542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:34.022411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:34.028561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e390e8-8b01-4940-8c42-dad953656971 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:34.061341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c81ac-0f9c-4c0d-ba10-227db22d104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:34.061923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c81ac-0f9c-4c0d-ba10-227db22d104c None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:34.062197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c81ac-0f9c-4c0d-ba10-227db22d104c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:34.062510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c81ac-0f9c-4c0d-ba10-227db22d104c None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c/users/dfbd17246ef94b98be996ca2ffeb36d1/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:34.064588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c81ac-0f9c-4c0d-ba10-227db22d104c None admin] RBAC: Authorizing `identity:create_grant(project_id=870dfee6c2a84684bd660d8e17688b4c, user_id=dfbd17246ef94b98be996ca2ffeb36d1, role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:34.066002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154c81ac-0f9c-4c0d-ba10-227db22d104c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:34.126617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 556/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:47:33 2022] PUT /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c/users/dfbd17246ef94b98be996ca2ffeb36d1/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:47:35.999731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962be528-53fa-44e2-9947-a42d19a80e24 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:36.000254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962be528-53fa-44e2-9947-a42d19a80e24 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:36.000254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962be528-53fa-44e2-9947-a42d19a80e24 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:36.001619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 568/1124] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:36.011338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04007-00a9-402b-9df3-ebce873dfb42 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:36.011836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04007-00a9-402b-9df3-ebce873dfb42 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:36.012303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04007-00a9-402b-9df3-ebce873dfb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:36.037339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-bac04007-00a9-402b-9df3-ebce873dfb42 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:36.038565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bac04007-00a9-402b-9df3-ebce873dfb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:36.104914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 557/1125] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:36.383778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962be528-53fa-44e2-9947-a42d19a80e24 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:36.467754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:36.483503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962be528-53fa-44e2-9947-a42d19a80e24 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:36.517180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:36.517789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:36.518064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:36.518362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] PATH_INFO: `/v3/roles/2dc56cca1e974e5fa8a42ddc39720152` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:36.519802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] RBAC: Authorizing `identity:get_role(role_id=2dc56cca1e974e5fa8a42ddc39720152)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:36.521157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:36.522490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 569/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:36 2022] GET /identity/v3/roles/2dc56cca1e974e5fa8a42ddc39720152 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:36.528956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac04007-00a9-402b-9df3-ebce873dfb42 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:36.609375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:36.623742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac04007-00a9-402b-9df3-ebce873dfb42 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:36.657324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:36.657880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:36.658156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:36.658453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] PATH_INFO: `/v3/users/dfbd17246ef94b98be996ca2ffeb36d1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:36.659547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] RBAC: Authorizing `identity:get_user(user_id=dfbd17246ef94b98be996ca2ffeb36d1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:36.660967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:36.662540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 558/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:36 2022] GET /identity/v3/users/dfbd17246ef94b98be996ca2ffeb36d1 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:36.668507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:36.766497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4a7a1-1ffd-415b-a118-5555b20e912e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:36.799340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:36.799932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:36.800212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:36.800484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:36.823986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:36.865439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:36.866916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:36.963248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 570/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:36.969246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:37.067136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190176-c517-4f81-bb7e-6c7610fce364 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:37.099829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b90c20-ea16-4887-a1bd-5262738fd228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:37.100450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b90c20-ea16-4887-a1bd-5262738fd228 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:37.100726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b90c20-ea16-4887-a1bd-5262738fd228 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:37.100994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b90c20-ea16-4887-a1bd-5262738fd228 None admin] PATH_INFO: `/v3/projects/870dfee6c2a84684bd660d8e17688b4c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:37.102084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b90c20-ea16-4887-a1bd-5262738fd228 None admin] RBAC: Authorizing `identity:get_project(project_id=870dfee6c2a84684bd660d8e17688b4c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:37.103579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b90c20-ea16-4887-a1bd-5262738fd228 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:37.105355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 559/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:36 2022] GET /identity/v3/projects/870dfee6c2a84684bd660d8e17688b4c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:37.111566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:37.165565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:37.209819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c2546-878b-4068-aa3e-f4d0237917b0 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:37.243763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a16b85-52c1-41f9-8af0-d1c2b14ea890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:37.244942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a16b85-52c1-41f9-8af0-d1c2b14ea890 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:37.245221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a16b85-52c1-41f9-8af0-d1c2b14ea890 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:37.245493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a16b85-52c1-41f9-8af0-d1c2b14ea890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:37.246213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a16b85-52c1-41f9-8af0-d1c2b14ea890 None admin] RBAC: Adding query filter params (role.id=2dc56cca1e974e5fa8a42ddc39720152, scope.project.id=870dfee6c2a84684bd660d8e17688b4c, user.id=dfbd17246ef94b98be996ca2ffeb36d1) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:37.246549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a16b85-52c1-41f9-8af0-d1c2b14ea890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:37.247973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a16b85-52c1-41f9-8af0-d1c2b14ea890 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:37.253598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 571/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:37 2022] GET /identity/v3/role_assignments?user.id=dfbd17246ef94b98be996ca2ffeb36d1&scope.project.id=870dfee6c2a84684bd660d8e17688b4c&role.id=2dc56cca1e974e5fa8a42ddc39720152 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:39.157925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590465-f714-4578-a868-a5c71d0e7984 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:39.158480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590465-f714-4578-a868-a5c71d0e7984 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:39.158480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590465-f714-4578-a868-a5c71d0e7984 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:39.159753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 560/1131] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:39.165084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae1319-0b0c-4eb3-bfbd-174a52803280 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:39.165518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae1319-0b0c-4eb3-bfbd-174a52803280 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:39.165620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae1319-0b0c-4eb3-bfbd-174a52803280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:39.184599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-66ae1319-0b0c-4eb3-bfbd-174a52803280 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:39.185790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-66ae1319-0b0c-4eb3-bfbd-174a52803280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:39.250476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 572/1132] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:39.520970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5590465-f714-4578-a868-a5c71d0e7984 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:39.545345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:39.620846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5590465-f714-4578-a868-a5c71d0e7984 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:39.654771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744dfe1-bafe-4731-a314-3271fd2d5ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:39.655403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744dfe1-bafe-4731-a314-3271fd2d5ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:39.655682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744dfe1-bafe-4731-a314-3271fd2d5ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:39.655952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744dfe1-bafe-4731-a314-3271fd2d5ca7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:39.657332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744dfe1-bafe-4731-a314-3271fd2d5ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:39.659199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744dfe1-bafe-4731-a314-3271fd2d5ca7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:39.660767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 561/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:39.667029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae1319-0b0c-4eb3-bfbd-174a52803280 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:39.714777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:39.763594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae1319-0b0c-4eb3-bfbd-174a52803280 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:39.796242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a11368-7e07-4439-b0a2-9fc4ab09f010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:39.796784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11368-7e07-4439-b0a2-9fc4ab09f010 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:39.797066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11368-7e07-4439-b0a2-9fc4ab09f010 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:39.797340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a11368-7e07-4439-b0a2-9fc4ab09f010 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:39.797978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a11368-7e07-4439-b0a2-9fc4ab09f010 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:39.799425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a11368-7e07-4439-b0a2-9fc4ab09f010 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:39.900218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 573/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:47:39 2022] POST /identity/v3/projects => generated 311 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:41.902876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f573-f408-4e16-a2ca-4778bf2c3c1b None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:41.903373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f573-f408-4e16-a2ca-4778bf2c3c1b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:41.903475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3f573-f408-4e16-a2ca-4778bf2c3c1b None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:41.904739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 562/1135] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:41.909884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab85889-05ed-4c11-9006-8c7cabf4d25b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:41.910341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab85889-05ed-4c11-9006-8c7cabf4d25b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:41.910529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab85889-05ed-4c11-9006-8c7cabf4d25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:41.927934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bab85889-05ed-4c11-9006-8c7cabf4d25b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:41.929106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bab85889-05ed-4c11-9006-8c7cabf4d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:41.993156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 574/1136] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:42.263754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3f573-f408-4e16-a2ca-4778bf2c3c1b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:42.350913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:42.360801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3f573-f408-4e16-a2ca-4778bf2c3c1b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:42.393747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcde72-cef1-473f-b4cd-3d4317f7f6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:42.394396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcde72-cef1-473f-b4cd-3d4317f7f6fc None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:42.394674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcde72-cef1-473f-b4cd-3d4317f7f6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:42.394940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcde72-cef1-473f-b4cd-3d4317f7f6fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:42.396024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcde72-cef1-473f-b4cd-3d4317f7f6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:42.397435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcde72-cef1-473f-b4cd-3d4317f7f6fc None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:42.398996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 563/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:47:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:42.405030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab85889-05ed-4c11-9006-8c7cabf4d25b None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:42.434671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:42.502739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab85889-05ed-4c11-9006-8c7cabf4d25b None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:42.536645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237afd-58a9-42c4-8862-b40441614cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:42.537216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237afd-58a9-42c4-8862-b40441614cc5 None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:42.537495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237afd-58a9-42c4-8862-b40441614cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:42.537768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237afd-58a9-42c4-8862-b40441614cc5 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:42.538436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237afd-58a9-42c4-8862-b40441614cc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:42.539861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237afd-58a9-42c4-8862-b40441614cc5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:42.600935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 575/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:47:42 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:44.528024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cabea6-b57f-4956-a330-04a3c7d12bdd None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:44.528588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cabea6-b57f-4956-a330-04a3c7d12bdd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:44.528588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cabea6-b57f-4956-a330-04a3c7d12bdd None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:44.529914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 564/1139] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:44.535247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126956ac-695f-46b2-9725-a11072b5f745 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:44.535676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126956ac-695f-46b2-9725-a11072b5f745 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:44.535801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126956ac-695f-46b2-9725-a11072b5f745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:44.553506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-126956ac-695f-46b2-9725-a11072b5f745 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:44.554729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-126956ac-695f-46b2-9725-a11072b5f745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:44.619874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 576/1140] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:44.885543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cabea6-b57f-4956-a330-04a3c7d12bdd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:44.984210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cabea6-b57f-4956-a330-04a3c7d12bdd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:45.016886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:45.017513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:45.017792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:45.018062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:45.028406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:45.033050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:45.034194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:45.035303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 565/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:47:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:45.040152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126956ac-695f-46b2-9725-a11072b5f745 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:45.057812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:45.136814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126956ac-695f-46b2-9725-a11072b5f745 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:45.170307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:45.170900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:45.171177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:45.171446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:45.172092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:45.172389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:45.173745 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:45.181427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 577/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:47:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:45.187290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:45.206825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:45.285450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2baeac8-6a9e-4f33-9b89-f31341be1b5b None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:45.317595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:45.318208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:45.318515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:45.318785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] PATH_INFO: `/v3/users/004ee9ac3b66424ba1d6d946d7216778` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:45.331239 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] RBAC: Authorizing `identity:get_user(user_id=004ee9ac3b66424ba1d6d946d7216778)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:45.332643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:45.334210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 566/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:45 2022] GET /identity/v3/users/004ee9ac3b66424ba1d6d946d7216778 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:45.340778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:45.435130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:45.456300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e2a7f-2483-415e-b3d8-8fb03753facd None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:45.489049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:45.489648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:45.489924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:45.490193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:45.555812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:45.557363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:45.642125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:45.668490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 578/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:45.674628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:45.750547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:45.777388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb0252-5cbe-4580-a47b-5c194a0378e1 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:45.811656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92d306-7100-4152-9f8d-8e80ca5c87f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:45.812285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d306-7100-4152-9f8d-8e80ca5c87f9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:45.812567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d306-7100-4152-9f8d-8e80ca5c87f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:45.812843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d306-7100-4152-9f8d-8e80ca5c87f9 None admin] PATH_INFO: `/v3/projects/8657a74cf1104beb8b85d8defcc66fe4` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:45.813950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92d306-7100-4152-9f8d-8e80ca5c87f9 None admin] RBAC: Authorizing `identity:get_project(project_id=8657a74cf1104beb8b85d8defcc66fe4)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:45.815421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92d306-7100-4152-9f8d-8e80ca5c87f9 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:45.817046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 567/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:45 2022] GET /identity/v3/projects/8657a74cf1104beb8b85d8defcc66fe4 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:45.823858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:45.918365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15419e1f-588b-49af-8052-3071c6e26447 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:45.951114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad0e70c-5803-4b29-98e8-5d62fbc2e775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:45.951695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0e70c-5803-4b29-98e8-5d62fbc2e775 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:45.951967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0e70c-5803-4b29-98e8-5d62fbc2e775 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:45.952233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad0e70c-5803-4b29-98e8-5d62fbc2e775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:45.952936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0e70c-5803-4b29-98e8-5d62fbc2e775 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=8657a74cf1104beb8b85d8defcc66fe4, user.id=004ee9ac3b66424ba1d6d946d7216778) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:45.953235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0e70c-5803-4b29-98e8-5d62fbc2e775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:45.954666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad0e70c-5803-4b29-98e8-5d62fbc2e775 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:45.959728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 579/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:45 2022] GET /identity/v3/role_assignments?user.id=004ee9ac3b66424ba1d6d946d7216778&scope.project.id=8657a74cf1104beb8b85d8defcc66fe4&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:47.819605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050377b4-182c-4b75-946c-f699b104c38e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:47.820152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050377b4-182c-4b75-946c-f699b104c38e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:47.820152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050377b4-182c-4b75-946c-f699b104c38e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:47.822078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 568/1147] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:47.827416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e399e-749c-48cc-a784-766f2b2a65a6 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:47.827844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e399e-749c-48cc-a784-766f2b2a65a6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:47.827958 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e399e-749c-48cc-a784-766f2b2a65a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:47.846040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-538e399e-749c-48cc-a784-766f2b2a65a6 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:47.847253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-538e399e-749c-48cc-a784-766f2b2a65a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:47.881085 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:47.911882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 580/1148] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:47:48.175032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050377b4-182c-4b75-946c-f699b104c38e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:48.273356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050377b4-182c-4b75-946c-f699b104c38e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:48.305244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:48.306413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:48.306685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:48.306952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:48.316593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:48.321171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:48.322411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:48.323435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 569/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:47:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:48.328258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e399e-749c-48cc-a784-766f2b2a65a6 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:48.422881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e399e-749c-48cc-a784-766f2b2a65a6 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:48.455072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:48.455610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:48.455886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:48.456153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:48.456804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:48.457101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:48.458471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:48.466166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 581/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:47:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:47:48.471795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:48.476799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:48.571192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d75dc9-d3a5-4131-afd7-3760ba6991cf None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:48.603646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:48.604273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:48.604552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:48.604820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] PATH_INFO: `/v3/users/004ee9ac3b66424ba1d6d946d7216778` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:48.605929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] RBAC: Authorizing `identity:get_user(user_id=004ee9ac3b66424ba1d6d946d7216778)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:48.607376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:48.608962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 570/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:48 2022] GET /identity/v3/users/004ee9ac3b66424ba1d6d946d7216778 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:48.614889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:48.649185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:48.714559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a003-2637-45b4-96a5-91ce7e992367 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:48.747881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248be555-922b-47a9-be47-ef9e71b7d25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:48.748491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248be555-922b-47a9-be47-ef9e71b7d25a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:48.748767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248be555-922b-47a9-be47-ef9e71b7d25a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:48.749036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248be555-922b-47a9-be47-ef9e71b7d25a None admin] PATH_INFO: `/v3/projects/8657a74cf1104beb8b85d8defcc66fe4` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:48.750123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248be555-922b-47a9-be47-ef9e71b7d25a None admin] RBAC: Authorizing `identity:get_project(project_id=8657a74cf1104beb8b85d8defcc66fe4)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:48.751615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248be555-922b-47a9-be47-ef9e71b7d25a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:48.753204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 582/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:48 2022] GET /identity/v3/projects/8657a74cf1104beb8b85d8defcc66fe4 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:48.759532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:48.826655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:48.855107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb6fdd-88bb-44a4-8682-5cc3ccfa9897 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:48.887785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fed02-bd2b-46e0-9f7d-a1f4d531b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:48.888438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fed02-bd2b-46e0-9f7d-a1f4d531b338 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:48.888711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fed02-bd2b-46e0-9f7d-a1f4d531b338 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:48.888977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fed02-bd2b-46e0-9f7d-a1f4d531b338 None admin] PATH_INFO: `/v3/projects/8657a74cf1104beb8b85d8defcc66fe4/users/004ee9ac3b66424ba1d6d946d7216778/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:48.890908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fed02-bd2b-46e0-9f7d-a1f4d531b338 None admin] RBAC: Authorizing `identity:create_grant(project_id=8657a74cf1104beb8b85d8defcc66fe4, user_id=004ee9ac3b66424ba1d6d946d7216778, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:48.892326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fed02-bd2b-46e0-9f7d-a1f4d531b338 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:48.977381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 571/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:47:48 2022] PUT /identity/v3/projects/8657a74cf1104beb8b85d8defcc66fe4/users/004ee9ac3b66424ba1d6d946d7216778/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:47:50.839687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57718-f194-4206-8358-332f2fafbcdf None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:50.840242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57718-f194-4206-8358-332f2fafbcdf None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:50.840242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57718-f194-4206-8358-332f2fafbcdf None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:50.841561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 583/1154] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:50.846732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965d69-24f8-4cd0-abb0-449daf7ea617 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:50.847106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965d69-24f8-4cd0-abb0-449daf7ea617 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:50.847280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b965d69-24f8-4cd0-abb0-449daf7ea617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:50.865307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9b965d69-24f8-4cd0-abb0-449daf7ea617 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:50.866530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9b965d69-24f8-4cd0-abb0-449daf7ea617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:50.932131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 572/1155] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:51.202660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57718-f194-4206-8358-332f2fafbcdf None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:51.299815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57718-f194-4206-8358-332f2fafbcdf None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:51.301288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:51.332517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:51.333133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:51.333411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:51.333679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:51.343470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:47:51.348208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:51.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:47:51.349922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 584/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:47:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:51.355460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965d69-24f8-4cd0-abb0-449daf7ea617 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:51.414164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:51.451773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b965d69-24f8-4cd0-abb0-449daf7ea617 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:51.484653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:51.485252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:51.485528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:51.485798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:51.486489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:51.486787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:51.488144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:51.496892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 573/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:47:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:51.503393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:51.600283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884a8e-1704-4ab0-96d1-8b73e3b06eab None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:51.633559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:51.634870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:51.634870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:51.634870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] PATH_INFO: `/v3/users/004ee9ac3b66424ba1d6d946d7216778` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:51.635911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] RBAC: Authorizing `identity:get_user(user_id=004ee9ac3b66424ba1d6d946d7216778)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:51.637322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:51.638910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 585/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:47:51 2022] GET /identity/v3/users/004ee9ac3b66424ba1d6d946d7216778 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:51.644857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:51.744956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b911060-da37-4685-91cf-b92a3acb590d None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:51.767541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:51.778362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:51.778968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:51.779251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:51.779524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:51.844665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:51.846103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:51.941924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 574/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:51.948018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:51.978457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:52.044862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42164ec2-950d-4ac2-96df-e244a2658737 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:52.076520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015c1396-895e-4dbb-acba-11a4cb4d5711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:52.077140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1396-895e-4dbb-acba-11a4cb4d5711 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:52.077418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1396-895e-4dbb-acba-11a4cb4d5711 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:52.077687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c1396-895e-4dbb-acba-11a4cb4d5711 None admin] PATH_INFO: `/v3/projects/8657a74cf1104beb8b85d8defcc66fe4` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:52.078802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c1396-895e-4dbb-acba-11a4cb4d5711 None admin] RBAC: Authorizing `identity:get_project(project_id=8657a74cf1104beb8b85d8defcc66fe4)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:52.080211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015c1396-895e-4dbb-acba-11a4cb4d5711 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:52.081811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 586/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:47:51 2022] GET /identity/v3/projects/8657a74cf1104beb8b85d8defcc66fe4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:52.088422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:52.112828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:52.185700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf119b-536a-4f52-8f63-783c06d11dd8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:52.219567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1d3fc-0082-4f12-a9c7-974333daf09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:52.220168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1d3fc-0082-4f12-a9c7-974333daf09c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:52.220447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1d3fc-0082-4f12-a9c7-974333daf09c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:52.220717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1d3fc-0082-4f12-a9c7-974333daf09c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:52.221429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1d3fc-0082-4f12-a9c7-974333daf09c None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=8657a74cf1104beb8b85d8defcc66fe4, user.id=004ee9ac3b66424ba1d6d946d7216778) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:52.221734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1d3fc-0082-4f12-a9c7-974333daf09c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:52.223178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1d3fc-0082-4f12-a9c7-974333daf09c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:52.228839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 575/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:47:52 2022] GET /identity/v3/role_assignments?user.id=004ee9ac3b66424ba1d6d946d7216778&scope.project.id=8657a74cf1104beb8b85d8defcc66fe4&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:54.147154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8560c-eac0-4df4-a25f-34e0d797ef33 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:54.147658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8560c-eac0-4df4-a25f-34e0d797ef33 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:54.147658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8560c-eac0-4df4-a25f-34e0d797ef33 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:54.148982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 587/1162] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:54.154388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67cb8f-029d-4c19-a293-729f88dcfe24 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:54.154769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67cb8f-029d-4c19-a293-729f88dcfe24 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:54.154967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67cb8f-029d-4c19-a293-729f88dcfe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:54.173175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0c67cb8f-029d-4c19-a293-729f88dcfe24 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:54.174415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0c67cb8f-029d-4c19-a293-729f88dcfe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:54.238970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 576/1163] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:54.516198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8560c-eac0-4df4-a25f-34e0d797ef33 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:54.611171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:54.615688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8560c-eac0-4df4-a25f-34e0d797ef33 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:54.648794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:54.649481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:54.649762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:54.650033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:54.713656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:54.715097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:54.807386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:54.811605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 588/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:47:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:47:54.817505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67cb8f-029d-4c19-a293-729f88dcfe24 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:54.835170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:54.913892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67cb8f-029d-4c19-a293-729f88dcfe24 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:54.948827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee6218-6987-4948-ab7c-80294c010a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:54.949473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee6218-6987-4948-ab7c-80294c010a98 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:54.949752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee6218-6987-4948-ab7c-80294c010a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:54.950021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee6218-6987-4948-ab7c-80294c010a98 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:54.956859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee6218-6987-4948-ab7c-80294c010a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:54.958310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee6218-6987-4948-ab7c-80294c010a98 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-edee6218-6987-4948-ab7c-80294c010a98 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:47:54.969820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:47:54.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:47:54.972330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 577/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:47:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:47:54.977121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:55.076823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bdb0a-d74d-4e00-87b1-28764b30e5f3 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:55.110520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb4864-317f-442e-8fe5-caf29ac69b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:55.111126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb4864-317f-442e-8fe5-caf29ac69b9d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:55.111411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb4864-317f-442e-8fe5-caf29ac69b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:55.111679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb4864-317f-442e-8fe5-caf29ac69b9d None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:55.112316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb4864-317f-442e-8fe5-caf29ac69b9d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:47:55.112614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb4864-317f-442e-8fe5-caf29ac69b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:55.113958 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb4864-317f-442e-8fe5-caf29ac69b9d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:55.121237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 589/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 22 10:47:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:47:57.089431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a096ff-3100-424a-9375-f1fd939fe78c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:57.089971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a096ff-3100-424a-9375-f1fd939fe78c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:57.089971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a096ff-3100-424a-9375-f1fd939fe78c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:57.091370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 578/1167] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:57.096487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61f325-e62a-460f-b7ea-6e158c0f4c20 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:57.096899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61f325-e62a-460f-b7ea-6e158c0f4c20 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:57.097041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61f325-e62a-460f-b7ea-6e158c0f4c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:57.115163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-3b61f325-e62a-460f-b7ea-6e158c0f4c20 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:57.116283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3b61f325-e62a-460f-b7ea-6e158c0f4c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:57.170395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:57.180081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 590/1168] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:57.442800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a096ff-3100-424a-9375-f1fd939fe78c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:57.520106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:47:57.541519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a096ff-3100-424a-9375-f1fd939fe78c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:47:57.574138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29749323-21ff-4d8f-aa44-ec394b32ceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:47:57.574748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749323-21ff-4d8f-aa44-ec394b32ceaa None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:57.575032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749323-21ff-4d8f-aa44-ec394b32ceaa None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:57.575299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749323-21ff-4d8f-aa44-ec394b32ceaa None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:57.575949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749323-21ff-4d8f-aa44-ec394b32ceaa None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:47:57.577303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749323-21ff-4d8f-aa44-ec394b32ceaa None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:47:57.629715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 579/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Mon Aug 22 10:47:57 2022] POST /identity/v3/domains => generated 267 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:47:59.529940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265679-64b3-496b-8920-c3c49b24b371 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:59.530613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265679-64b3-496b-8920-c3c49b24b371 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:59.530613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265679-64b3-496b-8920-c3c49b24b371 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:59.531867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 591/1170] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:47:59.536959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295569a7-0a2b-48e2-b392-de68e5444231 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:47:59.537363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295569a7-0a2b-48e2-b392-de68e5444231 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:47:59.537513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295569a7-0a2b-48e2-b392-de68e5444231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:47:59.555773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-295569a7-0a2b-48e2-b392-de68e5444231 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:47:59.556953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-295569a7-0a2b-48e2-b392-de68e5444231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:47:59.622104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 580/1171] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:47:59.894413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265679-64b3-496b-8920-c3c49b24b371 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:47:59.991360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265679-64b3-496b-8920-c3c49b24b371 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:00.024310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e568-2ead-47b3-885d-264b2254e84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:00.024913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e568-2ead-47b3-885d-264b2254e84b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:00.025189 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e568-2ead-47b3-885d-264b2254e84b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:00.025455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e568-2ead-47b3-885d-264b2254e84b None admin] PATH_INFO: `/v3/domains/1d3e75ac70ab45678d1090754c2f19ac` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:00.035288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39e568-2ead-47b3-885d-264b2254e84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=1d3e75ac70ab45678d1090754c2f19ac)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:00.036703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39e568-2ead-47b3-885d-264b2254e84b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:00.038249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 592/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 22 10:47:59 2022] GET /identity/v3/domains/1d3e75ac70ab45678d1090754c2f19ac => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:00.044758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295569a7-0a2b-48e2-b392-de68e5444231 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:00.109224 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:00.140296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295569a7-0a2b-48e2-b392-de68e5444231 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:00.174386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1d8c3-c876-41c3-bb40-2745351fa53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:00.174986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d8c3-c876-41c3-bb40-2745351fa53f None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:00.175265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d8c3-c876-41c3-bb40-2745351fa53f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:00.175535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1d8c3-c876-41c3-bb40-2745351fa53f None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:00.176165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1d8c3-c876-41c3-bb40-2745351fa53f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:00.177586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1d8c3-c876-41c3-bb40-2745351fa53f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:00.292458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 581/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:48:00 2022] POST /identity/v3/projects => generated 361 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:48:02.245853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef698df-feda-4fd4-9274-72cdaff2ca69 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:02.246400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef698df-feda-4fd4-9274-72cdaff2ca69 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:02.246515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef698df-feda-4fd4-9274-72cdaff2ca69 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:02.247761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 593/1174] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:48:02.252961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e1491-b3c4-4f62-81a7-06a2677e6691 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:02.253361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e1491-b3c4-4f62-81a7-06a2677e6691 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:02.253520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e1491-b3c4-4f62-81a7-06a2677e6691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:02.271962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-726e1491-b3c4-4f62-81a7-06a2677e6691 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:48:02.273149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-726e1491-b3c4-4f62-81a7-06a2677e6691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:48:02.338519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 582/1175] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:48:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:48:02.606588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef698df-feda-4fd4-9274-72cdaff2ca69 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:02.660394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:02.708079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef698df-feda-4fd4-9274-72cdaff2ca69 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:02.740780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8690c-c971-4933-8bd9-2f1e89fff4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:02.741392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8690c-c971-4933-8bd9-2f1e89fff4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:02.741666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8690c-c971-4933-8bd9-2f1e89fff4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:02.741993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8690c-c971-4933-8bd9-2f1e89fff4a0 None admin] PATH_INFO: `/v3/domains/1d3e75ac70ab45678d1090754c2f19ac` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:02.743474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8690c-c971-4933-8bd9-2f1e89fff4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1d3e75ac70ab45678d1090754c2f19ac)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:02.744896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8690c-c971-4933-8bd9-2f1e89fff4a0 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:02.746477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 594/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 22 10:48:02 2022] GET /identity/v3/domains/1d3e75ac70ab45678d1090754c2f19ac => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:02.752130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e1491-b3c4-4f62-81a7-06a2677e6691 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:02.799751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:02.846898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e1491-b3c4-4f62-81a7-06a2677e6691 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:02.880629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba304583-203a-4ae8-a909-1f84a93a5935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:02.881240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba304583-203a-4ae8-a909-1f84a93a5935 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:02.881518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba304583-203a-4ae8-a909-1f84a93a5935 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:02.881790 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba304583-203a-4ae8-a909-1f84a93a5935 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:02.882458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba304583-203a-4ae8-a909-1f84a93a5935 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:02.884496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba304583-203a-4ae8-a909-1f84a93a5935 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:02.965834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 583/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:48:02 2022] POST /identity/v3/users => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:48:04.895619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86fea-af46-47be-8221-9bd5134b4aee None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:04.896150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86fea-af46-47be-8221-9bd5134b4aee None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:04.896150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86fea-af46-47be-8221-9bd5134b4aee None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:04.897484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 595/1178] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:48:04.903399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e47149-780c-48c5-9834-6c249bbfbaa0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:04.903778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e47149-780c-48c5-9834-6c249bbfbaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:04.903937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e47149-780c-48c5-9834-6c249bbfbaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:04.922223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-75e47149-780c-48c5-9834-6c249bbfbaa0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:48:04.923433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-75e47149-780c-48c5-9834-6c249bbfbaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:48:04.988093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 584/1179] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:48:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:48:05.254415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d86fea-af46-47be-8221-9bd5134b4aee None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:05.352288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d86fea-af46-47be-8221-9bd5134b4aee None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:05.371675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:05.385545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:05.386161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:05.386471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:05.392330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:05.397864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:48:05.402542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:48:05.403614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:48:05.404457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 596/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:48:05.409778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e47149-780c-48c5-9834-6c249bbfbaa0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:05.443222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:05.505588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e47149-780c-48c5-9834-6c249bbfbaa0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:05.538805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:05.539478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:05.539760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:05.540034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:05.540691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:48:05.540992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:05.542381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:05.550131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 585/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:48:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:05.556521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:05.657228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e583cb7-1076-4b58-9ea3-b635be700e66 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:05.690614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:05.691236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:05.691516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:05.691791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] PATH_INFO: `/v3/users/9eae68d108c14c9e8c10e5ce9bf13d1c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:05.704776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] RBAC: Authorizing `identity:get_user(user_id=9eae68d108c14c9e8c10e5ce9bf13d1c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:05.706194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:05.707799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 597/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:48:05 2022] GET /identity/v3/users/9eae68d108c14c9e8c10e5ce9bf13d1c => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:05.713651 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:05.812932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:05.816610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90113180-2a7b-48de-8dc5-ba48d40585c5 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:05.849783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:05.850408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:05.850688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:05.850956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:05.916220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:05.917676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:06.008401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:06.012703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 586/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:48:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:48:06.018749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:06.058124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:06.115288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a3444-81f4-4e24-84ad-ed55122878a2 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:06.147674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f28eb-c515-451a-b691-3813bbd68bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:06.148941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f28eb-c515-451a-b691-3813bbd68bfe None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:06.149230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f28eb-c515-451a-b691-3813bbd68bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:06.149507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f28eb-c515-451a-b691-3813bbd68bfe None admin] PATH_INFO: `/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:06.150646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f28eb-c515-451a-b691-3813bbd68bfe None admin] RBAC: Authorizing `identity:get_project(project_id=a840c1a7275f4ca58dfd7ffcdcaa47f2)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:06.152091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f28eb-c515-451a-b691-3813bbd68bfe None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:06.153700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 598/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:48:06 2022] GET /identity/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:06.159967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:06.255479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5464498-7822-43e8-94e3-bb592aa79268 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:06.287935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52d509-e0cb-405f-bc54-5c5bcad7d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:06.289089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52d509-e0cb-405f-bc54-5c5bcad7d26c None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:06.289378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52d509-e0cb-405f-bc54-5c5bcad7d26c None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:06.289738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52d509-e0cb-405f-bc54-5c5bcad7d26c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:06.290483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52d509-e0cb-405f-bc54-5c5bcad7d26c None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=a840c1a7275f4ca58dfd7ffcdcaa47f2, user.id=9eae68d108c14c9e8c10e5ce9bf13d1c) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:48:06.290793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52d509-e0cb-405f-bc54-5c5bcad7d26c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:06.292207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52d509-e0cb-405f-bc54-5c5bcad7d26c None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:06.297363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 587/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:48:06 2022] GET /identity/v3/role_assignments?user.id=9eae68d108c14c9e8c10e5ce9bf13d1c&scope.project.id=a840c1a7275f4ca58dfd7ffcdcaa47f2&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:08.197329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556c004-d0b9-4be9-9dcd-34894d726eca None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:08.197877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556c004-d0b9-4be9-9dcd-34894d726eca None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:08.197877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556c004-d0b9-4be9-9dcd-34894d726eca None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:08.199146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 599/1186] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:48:08.204270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6be7d-9808-40d0-bfdd-318810a224fc None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:08.204675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6be7d-9808-40d0-bfdd-318810a224fc None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:08.204813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6be7d-9808-40d0-bfdd-318810a224fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:08.222930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-40a6be7d-9808-40d0-bfdd-318810a224fc None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:48:08.224100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-40a6be7d-9808-40d0-bfdd-318810a224fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:48:08.278472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:08.288125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 588/1187] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:48:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:48:08.562677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556c004-d0b9-4be9-9dcd-34894d726eca None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:08.661725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556c004-d0b9-4be9-9dcd-34894d726eca None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:08.669923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:08.696038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:08.696661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:08.696940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:08.697212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:08.706942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:48:08.712170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:48:08.713369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:48:08.714321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 600/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:48:08.719388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6be7d-9808-40d0-bfdd-318810a224fc None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:08.814887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6be7d-9808-40d0-bfdd-318810a224fc None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:08.847888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:08.848489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:08.848766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:08.849043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:08.849688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:48:08.849987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:08.851581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:08.859755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 589/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:48:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:08.865484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:08.961858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a441b9-22ce-470b-8ce0-87a816879618 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:08.993957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:08.994588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:08.994864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:08.995134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] PATH_INFO: `/v3/users/9eae68d108c14c9e8c10e5ce9bf13d1c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:08.996230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] RBAC: Authorizing `identity:get_user(user_id=9eae68d108c14c9e8c10e5ce9bf13d1c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:08.997651 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:08.999210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 601/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:48:08 2022] GET /identity/v3/users/9eae68d108c14c9e8c10e5ce9bf13d1c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:09.004873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:09.059592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:09.102692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8212-6e8e-484d-865f-f85d080f8a23 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:09.135553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ad91a-bcc8-4d0e-952d-edeba9a795f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:09.136175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ad91a-bcc8-4d0e-952d-edeba9a795f1 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:09.136454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ad91a-bcc8-4d0e-952d-edeba9a795f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:09.136729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ad91a-bcc8-4d0e-952d-edeba9a795f1 None admin] PATH_INFO: `/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:09.137820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ad91a-bcc8-4d0e-952d-edeba9a795f1 None admin] RBAC: Authorizing `identity:get_project(project_id=a840c1a7275f4ca58dfd7ffcdcaa47f2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:09.139269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ad91a-bcc8-4d0e-952d-edeba9a795f1 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:09.140866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 590/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:48:09 2022] GET /identity/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:09.146695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:09.179857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:09.243691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a818f1e-a345-4e60-a8f6-1e7e4a5af477 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:09.276743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64576df-d1ea-4be3-8aec-eef255102907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:09.277377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64576df-d1ea-4be3-8aec-eef255102907 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:09.277649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64576df-d1ea-4be3-8aec-eef255102907 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:09.277915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64576df-d1ea-4be3-8aec-eef255102907 None admin] PATH_INFO: `/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2/users/9eae68d108c14c9e8c10e5ce9bf13d1c/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:09.279789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64576df-d1ea-4be3-8aec-eef255102907 None admin] RBAC: Authorizing `identity:create_grant(project_id=a840c1a7275f4ca58dfd7ffcdcaa47f2, user_id=9eae68d108c14c9e8c10e5ce9bf13d1c, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:09.281274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64576df-d1ea-4be3-8aec-eef255102907 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:09.336342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 602/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:48:09 2022] PUT /identity/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2/users/9eae68d108c14c9e8c10e5ce9bf13d1c/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:48:11.203067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ea19b-4574-4567-985d-7ee7fac02bb7 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:11.203589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ea19b-4574-4567-985d-7ee7fac02bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:11.203589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ea19b-4574-4567-985d-7ee7fac02bb7 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:11.204906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 591/1193] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:48:11.210018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec732338-3d23-437b-a5a8-35a91e23363c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:11.210507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec732338-3d23-437b-a5a8-35a91e23363c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:11.210782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec732338-3d23-437b-a5a8-35a91e23363c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:11.228666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ec732338-3d23-437b-a5a8-35a91e23363c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:48:11.229850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ec732338-3d23-437b-a5a8-35a91e23363c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:48:11.308716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 603/1194] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:48:11.584816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ea19b-4574-4567-985d-7ee7fac02bb7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:11.691655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ea19b-4574-4567-985d-7ee7fac02bb7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:11.713268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:11.724318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:11.724946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:11.725223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:11.725491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:11.735690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:48:11.740315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:48:11.742218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:48:11.743094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 592/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:48:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:48:11.748517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec732338-3d23-437b-a5a8-35a91e23363c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:11.774823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:11.846764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec732338-3d23-437b-a5a8-35a91e23363c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:11.879407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:11.880011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:11.880293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:11.880566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:11.881227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:48:11.881527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:11.883627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:11.891388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 604/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:48:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:11.897146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:11.997602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6e62-50e5-494e-b7d1-a89410244daf None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:12.031107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:12.031731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:12.032009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:12.032278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] PATH_INFO: `/v3/users/9eae68d108c14c9e8c10e5ce9bf13d1c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:12.033369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] RBAC: Authorizing `identity:get_user(user_id=9eae68d108c14c9e8c10e5ce9bf13d1c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:12.034820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:12.036374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 593/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 22 10:48:11 2022] GET /identity/v3/users/9eae68d108c14c9e8c10e5ce9bf13d1c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:12.042476 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:12.125428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:12.139987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64798d27-83cb-4b80-9939-9fc828058d96 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:12.171654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:12.172250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:12.172529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:12.172801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:12.236812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:12.238253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:12.322206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:12.337359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 605/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:48:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:48:12.343328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:12.392032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:12.439080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04e62b-4436-487e-952d-1eceb930df7f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:12.471540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fa8c6-8a91-428c-859a-b5998801dca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:12.472172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa8c6-8a91-428c-859a-b5998801dca0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:12.472442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa8c6-8a91-428c-859a-b5998801dca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:12.472714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa8c6-8a91-428c-859a-b5998801dca0 None admin] PATH_INFO: `/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:12.473818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fa8c6-8a91-428c-859a-b5998801dca0 None admin] RBAC: Authorizing `identity:get_project(project_id=a840c1a7275f4ca58dfd7ffcdcaa47f2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:12.475283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fa8c6-8a91-428c-859a-b5998801dca0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:12.477534 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 594/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 22 10:48:12 2022] GET /identity/v3/projects/a840c1a7275f4ca58dfd7ffcdcaa47f2 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:12.483580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:48:12.579084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e27408-9dcc-419c-bc70-0cc21d1cf36b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:48:12.611812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47870162-b75b-4ec1-a0d9-cc55ff87ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:48:12.612407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47870162-b75b-4ec1-a0d9-cc55ff87ee2b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:48:12.612686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47870162-b75b-4ec1-a0d9-cc55ff87ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:48:12.612957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47870162-b75b-4ec1-a0d9-cc55ff87ee2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:48:12.613676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47870162-b75b-4ec1-a0d9-cc55ff87ee2b None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=a840c1a7275f4ca58dfd7ffcdcaa47f2, user.id=9eae68d108c14c9e8c10e5ce9bf13d1c) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:48:12.613981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47870162-b75b-4ec1-a0d9-cc55ff87ee2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:48:12.615431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47870162-b75b-4ec1-a0d9-cc55ff87ee2b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:48:12.621926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 606/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:48:12 2022] GET /identity/v3/role_assignments?user.id=9eae68d108c14c9e8c10e5ce9bf13d1c&scope.project.id=a840c1a7275f4ca58dfd7ffcdcaa47f2&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:48:22.467398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=41 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:48:22.619372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=75 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:25.038822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e50046-10db-43fe-86ec-b52d63144bcd None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:25.039448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e50046-10db-43fe-86ec-b52d63144bcd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:25.039448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e50046-10db-43fe-86ec-b52d63144bcd None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:25.040706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 595/1201] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:25.046065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6ca6-50d4-4006-9073-7e3826971f4b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:25.046487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6ca6-50d4-4006-9073-7e3826971f4b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:25.046726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f6ca6-50d4-4006-9073-7e3826971f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:25.065148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-549f6ca6-50d4-4006-9073-7e3826971f4b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:25.066388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-549f6ca6-50d4-4006-9073-7e3826971f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:25.131192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 607/1202] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:49:25.401387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e50046-10db-43fe-86ec-b52d63144bcd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:25.500839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e50046-10db-43fe-86ec-b52d63144bcd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:25.534175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:25.534845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:25.535123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:25.535393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:25.545961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:25.547472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:25.562251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:25.563468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:25.564428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:25.564428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:25.564428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:25.564428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 596/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:25.569503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6ca6-50d4-4006-9073-7e3826971f4b None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:25.666903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f6ca6-50d4-4006-9073-7e3826971f4b None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:25.695150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:25.700975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44008a59-179e-415e-844c-c4eda527b5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:25.701572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44008a59-179e-415e-844c-c4eda527b5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:25.701849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44008a59-179e-415e-844c-c4eda527b5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:25.702118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44008a59-179e-415e-844c-c4eda527b5d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:25.702809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44008a59-179e-415e-844c-c4eda527b5d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:25.703105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44008a59-179e-415e-844c-c4eda527b5d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:25.704454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44008a59-179e-415e-844c-c4eda527b5d1 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:25.715532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 608/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:25.721454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:25.774499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:25.821850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ec8f6-2a54-4e3d-aa9c-1c8fe4a25c31 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:25.855103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d67f8-554b-4538-9f22-b206aa8ba084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:25.855719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d67f8-554b-4538-9f22-b206aa8ba084 None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:25.856001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d67f8-554b-4538-9f22-b206aa8ba084 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:25.856273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d67f8-554b-4538-9f22-b206aa8ba084 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:25.856913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d67f8-554b-4538-9f22-b206aa8ba084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:25.858372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d67f8-554b-4538-9f22-b206aa8ba084 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:25.949503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 597/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 22 10:49:25 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:49:27.967216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d389-3677-4dc6-ba6f-345fd934a8af None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:27.967751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d389-3677-4dc6-ba6f-345fd934a8af None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:27.967751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d389-3677-4dc6-ba6f-345fd934a8af None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:27.969145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 609/1206] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:27.974885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08782e0b-2670-48d9-8ff1-5fb7348a40dd None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:27.975383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08782e0b-2670-48d9-8ff1-5fb7348a40dd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:27.975476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08782e0b-2670-48d9-8ff1-5fb7348a40dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:27.994086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-08782e0b-2670-48d9-8ff1-5fb7348a40dd None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:27.995363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-08782e0b-2670-48d9-8ff1-5fb7348a40dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:28.064604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 598/1207] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:28.329108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0d389-3677-4dc6-ba6f-345fd934a8af None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:28.430515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0d389-3677-4dc6-ba6f-345fd934a8af None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:28.463407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:28.464027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:28.464307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:28.464580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:28.475592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:49:28.480214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:28.481201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:28.482047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 610/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:49:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:28.487358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08782e0b-2670-48d9-8ff1-5fb7348a40dd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:28.526600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:28.586291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08782e0b-2670-48d9-8ff1-5fb7348a40dd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:28.622310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:28.623206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:28.623628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:28.624042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:28.625049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:28.625509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:28.627737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:28.640216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 599/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:49:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:28.646468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:28.697065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:28.745419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b5a60-c3d6-41ce-b9c1-fed5e1326ede None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:28.777197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:28.777826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:28.778107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:28.778411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:28.789253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:28.791283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:28.805975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:28.806975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:28.807841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:28.807841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:28.807841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:28.807841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 611/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:28.812981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:28.909737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:28.913334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e6fa-4ffb-49ea-bdb5-bf0b3414cc47 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:28.945988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:28.946625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:28.946950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:28.947340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:28.948305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:28.948762 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:28.950991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:28.967700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 600/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:28.973386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:29.078467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15056d-7788-4e3d-95eb-5a25c56f00bb None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:29.079988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:29.112650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:29.113232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:29.113496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:29.114030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:29.122172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:29.123648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:49:29.134331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:29.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:29.147709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 612/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:29.148407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:29.241232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce507b1d-efcb-4efe-a16a-43ccbc5d55b6 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:29.274630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:29.275245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:29.275529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:29.275803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:29.276527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:29.276831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:29.278277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:29.294848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 601/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 22 10:49:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:29.300675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:29.408124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049ce8e-bd51-4aa0-88ca-67fedbec21aa None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:29.440498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:29.441094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:29.441369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:29.441639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:29.471260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:29.508015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:29.509450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:29.603071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 613/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:49:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:49:29.608817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:29.627519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:29.711287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb8389-3cff-4ed7-9130-2fdc4045e0c8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:29.745553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:29.746184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:29.746490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:29.746760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:29.756973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:29.758432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:29.772551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:29.777167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:29.778375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:29.778375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:29.778375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:29.778375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 602/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:29.779529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:29.839412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:29.877176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45a619-5326-4aeb-964a-414aa3d2c265 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:29.911938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:29.912601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:29.912881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:29.913152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:29.913945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:29.914274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:29.915631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:29.926508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 614/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:29.932980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:30.001916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:30.031725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07578d52-64f5-4a0c-8e2b-7df45eb31e32 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:30.066939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:30.067687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:30.068001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:30.068274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:30.075155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:30.077825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:49:30.087816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:30.088884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:30.089958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 603/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:30.094972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:30.196113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379acd0-2eec-49ac-86b1-8b328a3e01a9 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:30.217858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:30.229680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fc300-41fc-48ca-bf1f-f010e41ba1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:30.230312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fc300-41fc-48ca-bf1f-f010e41ba1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:30.230593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fc300-41fc-48ca-bf1f-f010e41ba1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:30.230870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fc300-41fc-48ca-bf1f-f010e41ba1c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:30.231551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fc300-41fc-48ca-bf1f-f010e41ba1c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:30.231853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fc300-41fc-48ca-bf1f-f010e41ba1c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:30.233277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fc300-41fc-48ca-bf1f-f010e41ba1c1 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:30.245514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 615/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:30.251717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:30.350899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64adcb-afc0-42d5-bbb7-39877e0937a8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:30.379325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:30.385174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f5f01-ab1d-495f-b94f-ace395358b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:30.385778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5f01-ab1d-495f-b94f-ace395358b97 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:30.386057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5f01-ab1d-495f-b94f-ace395358b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:30.386360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f5f01-ab1d-495f-b94f-ace395358b97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:30.387078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f5f01-ab1d-495f-b94f-ace395358b97 None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=f3d2213b387f4938ae20eeb16c9352de) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:30.387382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f5f01-ab1d-495f-b94f-ace395358b97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:30.388800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f5f01-ab1d-495f-b94f-ace395358b97 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:30.393943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 604/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:49:30 2022] GET /identity/v3/role_assignments?user.id=f3d2213b387f4938ae20eeb16c9352de&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:32.255736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987e77-aebc-47d1-8d50-20675a73a4cf None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:32.256365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987e77-aebc-47d1-8d50-20675a73a4cf None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:32.256365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72987e77-aebc-47d1-8d50-20675a73a4cf None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:32.257723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 616/1220] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:32.262835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6f283-18b1-423c-8ec0-21bcca135efd None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:32.263287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6f283-18b1-423c-8ec0-21bcca135efd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:32.263383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6f283-18b1-423c-8ec0-21bcca135efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:32.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b7f6f283-18b1-423c-8ec0-21bcca135efd None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:32.283145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b7f6f283-18b1-423c-8ec0-21bcca135efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:32.349446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 605/1221] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:32.620083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987e77-aebc-47d1-8d50-20675a73a4cf None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:32.721600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72987e77-aebc-47d1-8d50-20675a73a4cf None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:32.756460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:32.757063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:32.757340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:32.757612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:32.767293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:49:32.771952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:32.773105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:32.774156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 617/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:49:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:32.779152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6f283-18b1-423c-8ec0-21bcca135efd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:32.879884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6f283-18b1-423c-8ec0-21bcca135efd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:32.904855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:32.915684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:32.916300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:32.916589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:32.916866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:32.917530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] RBAC: Adding query filter params (name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:32.917834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:32.919291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:32.928497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 606/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:49:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:32.934350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:32.976559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:33.032118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be50158-bb79-4a9e-957c-56ddf2a63815 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:33.065331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:33.065956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:33.066268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:33.066530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:33.077524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:33.079049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:33.092949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:33.094045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:33.095218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:33.095218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:33.095218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:33.095218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 618/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:33.100351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:33.199884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f59fd-f2d0-4d7c-af11-56fde3387097 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:33.232815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:33.233387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:33.233667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:33.233936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:33.234609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:33.234910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:33.236264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:33.249050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 607/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:49:33.254816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:33.345255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:33.352682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df1a7d-ce70-4092-ab8d-066775f2f931 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:33.385943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:33.386596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:33.386878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:33.387148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:33.394647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:33.397218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:49:33.408264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:33.409345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:33.410509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 619/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:33.416075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:33.465954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:33.518000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251a77-743e-4628-9449-0e68815ede9e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:33.551302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:33.551928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:33.552207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:33.552475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:33.553239 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:33.553538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:33.554978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:33.572421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 608/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 22 10:49:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:33.578422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:33.676405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad50c51-4170-43c3-811e-36602a5f45cc None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:33.709691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:33.710333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:33.710617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:33.710890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:33.721167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:33.722573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:33.740052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:33.741221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:33.742487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:33.742487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:33.742487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:33.742487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 620/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:33.748410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:33.847181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9c46a-b538-434f-8c8e-ee24a0b2d88a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:33.850244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:33.883727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:33.884382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:33.884775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:33.885045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:33.885678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:33.885976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:33.887367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:33.898617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 609/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:33.904330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:33.914535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:34.003429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1ac9c-ce82-40da-8d1c-a02a428f3e01 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:34.035725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:34.037154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:34.037433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:34.037702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:34.044091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:34.045516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:49:34.057912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:34.065099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:34.066324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 621/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:49:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:34.066933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:34.171959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1a4c-99e0-41ab-a4b4-f9a6f181782f None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:34.204707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae0bec-2ee1-4d29-9377-60de6cdf43a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:34.205315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae0bec-2ee1-4d29-9377-60de6cdf43a0 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:34.205594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae0bec-2ee1-4d29-9377-60de6cdf43a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:34.205867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae0bec-2ee1-4d29-9377-60de6cdf43a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:34.206593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae0bec-2ee1-4d29-9377-60de6cdf43a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:34.206895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae0bec-2ee1-4d29-9377-60de6cdf43a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:34.208855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae0bec-2ee1-4d29-9377-60de6cdf43a0 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:34.220285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 610/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:49:34.227135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:34.291659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:34.324983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cbb8c-26ca-48e6-99a2-a147fe717d8b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:34.357926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a618ef-c046-45d9-95e7-2687877dbd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:34.358605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a618ef-c046-45d9-95e7-2687877dbd14 None admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:34.358871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a618ef-c046-45d9-95e7-2687877dbd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:34.359147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a618ef-c046-45d9-95e7-2687877dbd14 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/f3d2213b387f4938ae20eeb16c9352de/roles/401c0ae4c0ad4b79bb877cbf745a71f2` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:34.373276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a618ef-c046-45d9-95e7-2687877dbd14 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=f3d2213b387f4938ae20eeb16c9352de, role_id=401c0ae4c0ad4b79bb877cbf745a71f2)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:34.374752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a618ef-c046-45d9-95e7-2687877dbd14 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:34.497775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 622/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:49:34 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/f3d2213b387f4938ae20eeb16c9352de/roles/401c0ae4c0ad4b79bb877cbf745a71f2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:49:36.379795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f600c21-ca70-41a1-a13a-dcca40f864cb None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:36.380320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f600c21-ca70-41a1-a13a-dcca40f864cb None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:36.380430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f600c21-ca70-41a1-a13a-dcca40f864cb None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:36.381811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 611/1233] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:36.387140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9ed8-197d-46fd-9893-2713d7532fd2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:36.390495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9ed8-197d-46fd-9893-2713d7532fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:36.390495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b9ed8-197d-46fd-9893-2713d7532fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:36.405935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4e5b9ed8-197d-46fd-9893-2713d7532fd2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:36.407655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4e5b9ed8-197d-46fd-9893-2713d7532fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:36.472021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 623/1234] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:36.744623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f600c21-ca70-41a1-a13a-dcca40f864cb None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:36.769924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:36.847347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f600c21-ca70-41a1-a13a-dcca40f864cb None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:36.882124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:36.882788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:36.883066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:36.883337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:36.893349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:49:36.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:36.900166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:36.901167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 612/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 22 10:49:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:36.906042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9ed8-197d-46fd-9893-2713d7532fd2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:36.951733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:37.002643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b9ed8-197d-46fd-9893-2713d7532fd2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:37.035959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:37.036530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:37.036811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:37.037082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:37.037740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:37.038042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:37.039483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:37.047058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 624/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 22 10:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:37.052903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:37.138404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:37.153858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b491c-8167-4959-b84b-aefbd916a0a9 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:37.186691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:37.187356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:37.187635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:37.187904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:37.198798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:37.200232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:37.213973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:37.215049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:37.216121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:37.216121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:37.216121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:37.216121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 613/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:37.221039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:37.321563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d890ca-a564-4654-b8e5-bf4d7416fc8f None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:37.330888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:37.356332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:37.356948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:37.357285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:37.357559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:37.358619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:37.358619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:37.359898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:37.373214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 625/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:37.378912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:37.479519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbcbe5-ad55-4098-b3fe-ab21d2db61b3 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:37.513221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:37.513841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:37.514116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:37.514436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:37.517112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:37.522044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:37.523526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:49:37.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:37.536343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:37.537606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 614/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:37.542343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:37.640186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66568edb-2bee-4c74-a91c-5b310e44ae17 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:37.674670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:37.675276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:37.675554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:37.675825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:37.677159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:37.677499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:37.678967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:37.695553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 626/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 22 10:49:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:37.701384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:37.802437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0403b-a557-4e41-90cb-311e37b1a15e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:37.840476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:37.841054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:37.841350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:37.841603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:37.904316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:37.911380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:37.912884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:38.008761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 615/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:49:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:49:38.014738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:38.042530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:38.119917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265cd13-33a5-4e32-a3de-475c141224c8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:38.154107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:38.154844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:38.155066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:38.155342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:38.165981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:38.167516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:38.181649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:38.182641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:38.183529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:38.183529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:38.183529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:38.183529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 627/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:38.188726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:38.281898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:38.287742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96c936-79f5-444a-b6e9-e754c2d1c45e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:38.321515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:38.322130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:38.322439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:38.322712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:38.323391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:38.323691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:38.325109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:38.338008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 616/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:38.343781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:38.421564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:38.442390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc3ad-c9c4-4233-91ca-2eec2b6d84ad None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:38.474291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:38.474943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:38.475216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:38.475482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:38.481749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:38.483200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:49:38.494813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:38.495803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:38.496656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 628/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:38.502278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:38.599435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6a885-74a6-4f7a-9b78-8062b08d9195 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:38.632539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27508d-ee1a-44f8-b922-68c36690095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:38.633178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27508d-ee1a-44f8-b922-68c36690095f None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:38.633455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27508d-ee1a-44f8-b922-68c36690095f None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:38.633723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27508d-ee1a-44f8-b922-68c36690095f None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:38.634434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27508d-ee1a-44f8-b922-68c36690095f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:38.634732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27508d-ee1a-44f8-b922-68c36690095f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:38.636151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27508d-ee1a-44f8-b922-68c36690095f None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:38.648440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 617/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:38.654631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:38.753244 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294199-34af-4a20-866a-bdd559eca38b None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:38.787113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f3b53-6913-46ec-88e5-85b1b6b9b3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:38.787686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f3b53-6913-46ec-88e5-85b1b6b9b3ab None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:38.787961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f3b53-6913-46ec-88e5-85b1b6b9b3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:38.788234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f3b53-6913-46ec-88e5-85b1b6b9b3ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:38.788942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f3b53-6913-46ec-88e5-85b1b6b9b3ab None admin] RBAC: Adding query filter params (role.id=401c0ae4c0ad4b79bb877cbf745a71f2, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=f3d2213b387f4938ae20eeb16c9352de) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:38.789243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f3b53-6913-46ec-88e5-85b1b6b9b3ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:38.790710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f3b53-6913-46ec-88e5-85b1b6b9b3ab None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:38.796786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 629/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:49:38 2022] GET /identity/v3/role_assignments?user.id=f3d2213b387f4938ae20eeb16c9352de&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=401c0ae4c0ad4b79bb877cbf745a71f2 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:40.766390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f7bb8-39ab-4002-a746-40c8bf006256 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:40.767023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f7bb8-39ab-4002-a746-40c8bf006256 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:40.767023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f7bb8-39ab-4002-a746-40c8bf006256 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:40.768309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 618/1247] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:40.773759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db96322-2e84-4202-825c-1606125c352a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:40.774154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db96322-2e84-4202-825c-1606125c352a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:40.774373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db96322-2e84-4202-825c-1606125c352a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:40.780662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:40.792935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6db96322-2e84-4202-825c-1606125c352a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:40.794124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6db96322-2e84-4202-825c-1606125c352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:40.859408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 630/1248] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:41.127526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f7bb8-39ab-4002-a746-40c8bf006256 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:41.146092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:41.230070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f7bb8-39ab-4002-a746-40c8bf006256 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:41.263957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:41.264581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:41.264859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:41.265131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:41.276889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:49:41.281924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:41.283135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:41.284488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 619/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:49:41.289354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db96322-2e84-4202-825c-1606125c352a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:41.388406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db96322-2e84-4202-825c-1606125c352a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:41.416503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:41.421098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:41.421692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:41.421969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:41.422252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:41.422917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:41.423216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:41.424568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:41.432496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 631/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:41.438200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:41.516227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:41.539071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1df88f-43e4-45f5-aec0-ed472262ee11 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:41.572788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:41.573426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:41.573715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:41.573991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:41.584653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:41.586091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:41.600600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:41.604981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:41.606074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:41.606074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:41.606074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:41.606074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 620/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:41.607736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:41.705534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2de5a4-1de7-4210-b183-9f79aa8ff9e5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:41.737020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:41.737612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:41.737885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:41.738444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:41.743102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:41.743102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:41.743102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:41.752707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 632/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:41.759034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:41.856901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f16aa-f213-4c7c-aef9-6ba760e8415a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:41.883735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:41.892410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:41.893089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:41.893368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:41.893637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:41.902753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:41.904180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:49:41.915310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:41.916457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:41.917798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 621/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:41.923808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:41.976213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:42.019910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501632c-a1cd-4e97-a488-767e9e7dcf15 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:42.053948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:42.054583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:42.054881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:42.055153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:42.055873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:42.056176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:42.057583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:42.074006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 633/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 22 10:49:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:42.079839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:42.188728 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11a33f-cdbd-494c-b3fb-58acddf2b08e None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:42.224181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:42.224782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:42.225059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:42.225459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:42.279315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:42.291693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:42.294048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:42.389939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 622/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:49:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:49:42.396592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:42.497927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd74e6-a9b0-4af4-84e4-9d4fd4723e5e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:42.511054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:42.531765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:42.532333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:42.532620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:42.532879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:42.543701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:42.545261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:42.560047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:42.561201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:42.562219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:42.562219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:42.562219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:42.562219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 634/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:42.566967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:42.654476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:42.666443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc9d4f-3ce2-464c-b4e4-7d7866642b18 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:42.698667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:42.699270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:42.699546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:42.699814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:42.700443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:42.700739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:42.702098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:42.713555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 623/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:42.719313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:42.819514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745cf37-e146-423c-8250-aab82c1cdb58 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:42.853630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:42.854284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:42.854560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:42.854916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:42.861633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:42.869427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:49:42.874288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:42.875258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:42.876231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 635/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:42.882012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:42.978908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65d926-60a4-4fc5-b196-4ee1f52029fb None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:43.012689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35782e90-7756-4302-8ffa-842115986e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:43.013296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782e90-7756-4302-8ffa-842115986e8d None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:43.013584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782e90-7756-4302-8ffa-842115986e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:43.013856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35782e90-7756-4302-8ffa-842115986e8d None admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:43.014676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35782e90-7756-4302-8ffa-842115986e8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:43.014995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35782e90-7756-4302-8ffa-842115986e8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:43.016410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35782e90-7756-4302-8ffa-842115986e8d None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:43.028921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 624/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:43.035159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:43.060217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:43.133440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46882955-ec0a-4432-870c-3b77e15337a4 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:43.166388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e134773-e50a-46c2-ba9d-ad5968dcaa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:43.166980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e134773-e50a-46c2-ba9d-ad5968dcaa14 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:43.167254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e134773-e50a-46c2-ba9d-ad5968dcaa14 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:43.167518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e134773-e50a-46c2-ba9d-ad5968dcaa14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:43.168222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e134773-e50a-46c2-ba9d-ad5968dcaa14 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=f3d2213b387f4938ae20eeb16c9352de) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:43.168522 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e134773-e50a-46c2-ba9d-ad5968dcaa14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:43.169917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e134773-e50a-46c2-ba9d-ad5968dcaa14 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:43.175170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 636/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:49:43 2022] GET /identity/v3/role_assignments?user.id=f3d2213b387f4938ae20eeb16c9352de&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:45.040628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c357-a52f-46ef-814f-1613534b8ba0 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:45.041186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c357-a52f-46ef-814f-1613534b8ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:45.041186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c357-a52f-46ef-814f-1613534b8ba0 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:45.042558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 625/1261] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:45.047816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1deb-77d0-431c-87ec-1a41c4f5aec1 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:45.048249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1deb-77d0-431c-87ec-1a41c4f5aec1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:45.048353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb1deb-77d0-431c-87ec-1a41c4f5aec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:45.066781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f4cb1deb-77d0-431c-87ec-1a41c4f5aec1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:45.067981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f4cb1deb-77d0-431c-87ec-1a41c4f5aec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:45.134716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 637/1262] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:45.402285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c357-a52f-46ef-814f-1613534b8ba0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:45.414189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:45.502054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c357-a52f-46ef-814f-1613534b8ba0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:45.534899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:45.536162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:45.536440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:45.536706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:45.547557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:49:45.552465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:45.553414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:49:45.554280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:45.558397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 626/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:49:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:45.559622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1deb-77d0-431c-87ec-1a41c4f5aec1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:45.577297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:45.655111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb1deb-77d0-431c-87ec-1a41c4f5aec1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:45.689383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:45.689979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:45.690279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:45.690552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:45.691224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:45.691618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:45.692971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:45.700923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 638/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:45.706579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:45.774825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:45.806782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10ad23-269e-49d9-849f-860eb7cf5083 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:45.839433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:45.840103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:45.840384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:45.840655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:45.851459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:45.855651 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:45.870021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:45.871333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:45.872513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:45.872513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:45.872513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:45.872513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 627/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:45.876555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:45.956823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:45.975667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7eb07-2e60-4fe7-9b90-426bed60a997 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:46.009160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:46.009756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:46.010036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:46.010352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:46.010998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:46.011299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:46.012666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:46.024470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 639/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:46.030114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:46.136662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad9f44-7b02-4710-8bca-278848a677b5 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:46.156400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:46.170016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:46.170667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:46.170987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:46.171302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:46.179183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:46.180701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:49:46.191979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:46.193095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:46.194339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 628/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:46.199111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:46.297031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b101ee-d4a8-4451-9b09-18bac77d673a None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:46.331369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:46.331973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:46.332400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:46.332655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:46.333228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:46.333529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:46.334981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:46.344235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:46.352440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 640/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 22 10:49:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:46.358324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:46.457087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618eb66-ebb0-49ec-a3a2-da3dad335edc None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:46.490481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:46.491107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:46.491382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:46.491655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:46.501773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:46.503236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:46.517427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:46.518632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:46.519719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:46.519719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:46.519719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:46.519719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 629/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 22 10:49:46.524949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:46.624530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef1e80-3410-47d0-8d91-3e6b9d7f9c8d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:46.657045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:46.657644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:46.657924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:46.658194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:46.658859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:46.659159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:46.661078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:46.671823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 641/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:46.678311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:46.709997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:46.778144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79193d60-57ec-4262-a8c1-d0980524b10a None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:46.812713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:46.813340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:46.813618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:46.813887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:46.820448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:46.821987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:49:46.832796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:46.833798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:46.834773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 630/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:46.839754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:46.900964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:46.940904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff2448-a133-4a85-bc85-6d4a8afa22ee None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:46.975847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc663b7-7227-42be-8ccd-7318d8d9946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:46.980545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc663b7-7227-42be-8ccd-7318d8d9946c None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:46.980545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc663b7-7227-42be-8ccd-7318d8d9946c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:46.980545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc663b7-7227-42be-8ccd-7318d8d9946c None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:46.980545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc663b7-7227-42be-8ccd-7318d8d9946c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:46.980545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc663b7-7227-42be-8ccd-7318d8d9946c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:46.980545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc663b7-7227-42be-8ccd-7318d8d9946c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:46.991560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 642/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:46.997568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:47.087503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:47.096031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d9c8f3-1bd6-4899-9659-2bf2d2db2822 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:47.129391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e45716-3ec4-43d4-9e64-56c0d153a2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:47.130045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e45716-3ec4-43d4-9e64-56c0d153a2a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:47.130378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e45716-3ec4-43d4-9e64-56c0d153a2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:47.130658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e45716-3ec4-43d4-9e64-56c0d153a2a8 None admin] PATH_INFO: `/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/f3d2213b387f4938ae20eeb16c9352de/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:47.132534 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e45716-3ec4-43d4-9e64-56c0d153a2a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e46a0cfe4c2b4edf9d5e27161f97e08c, user_id=f3d2213b387f4938ae20eeb16c9352de, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:47.133962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e45716-3ec4-43d4-9e64-56c0d153a2a8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:47.280094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 631/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 22 10:49:46 2022] PUT /identity/v3/projects/e46a0cfe4c2b4edf9d5e27161f97e08c/users/f3d2213b387f4938ae20eeb16c9352de/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 10:49:49.169706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecd456-22ef-4de0-a467-9753fb1e327c None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:49.170217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecd456-22ef-4de0-a467-9753fb1e327c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:49.170217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecd456-22ef-4de0-a467-9753fb1e327c None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:49.171587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 643/1274] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:49.176865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d6a3-d977-4ad1-8603-150a758d6eb2 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:49.177252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d6a3-d977-4ad1-8603-150a758d6eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:49.177413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91d6a3-d977-4ad1-8603-150a758d6eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:49.195571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9c91d6a3-d977-4ad1-8603-150a758d6eb2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:49.196763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9c91d6a3-d977-4ad1-8603-150a758d6eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:49.264002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 632/1275] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:49.542493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecd456-22ef-4de0-a467-9753fb1e327c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:49.645773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ecd456-22ef-4de0-a467-9753fb1e327c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:49.662783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:49.678868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:49.679489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:49.680150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:49.680150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:49.689583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 22 10:49:49.694995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:49.696096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:49.697333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 644/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 22 10:49:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:49.702222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91d6a3-d977-4ad1-8603-150a758d6eb2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:49.781577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:49.803186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91d6a3-d977-4ad1-8603-150a758d6eb2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:49.832893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:49.833498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:49.833775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:49.834043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:49.837643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:49.837643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:49.837643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:49.844635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 633/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 22 10:49:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:49.851012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:49.949784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8a93d-f46b-40c0-811e-eded8e06a17e None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:49.984030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:49.984695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:49.984973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:49.985244 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:49.995535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:49.997004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:50.011988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:50.013117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:50.014187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:50.014187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:50.014187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:50.014187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 645/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:50.019614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:50.118518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601f471-4c1b-48e2-bc5e-32738eb64475 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:50.160928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:50.161543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:50.161832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:50.162715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:50.163356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:50.163660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:50.165027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:50.177349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 634/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:50.183204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:50.214017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:50.281586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1b45-e698-4d1f-b50d-92c978dc05b8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:50.313564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:50.314177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:50.314517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:50.314782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:50.322788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:50.324297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 22 10:49:50.338555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:50.339692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:50.340953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 646/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:50.345452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:50.349552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:50.442254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a958f9-67c1-4e47-83b1-391bca5ae3da None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:50.476625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:50.477233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:50.477512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:50.477783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:50.478570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:50.478859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:50.480276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:50.497515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 635/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 22 10:49:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:50.503364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:50.599793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:50.609491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f569582-e935-4062-8bbf-e860172653e8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:50.643230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:50.643867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:50.644800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:50.645081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:50.712119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:50.713561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:50.801575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:50.811569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 647/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 22 10:49:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 22 10:49:50.818448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:50.881141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:50.919918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ba899-4aac-4511-9f56-54385c5ddc74 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:50.958061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:50.958750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:50.959025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:50.959294 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:50.972586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:50.974062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 22 10:49:50.992320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:50.993516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 22 10:49:50.994509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 22 10:49:50.994509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 22 10:49:50.994509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:50.998518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 636/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:50.999820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:51.107678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d66f3d-e071-4ad8-a949-491c1ddda7f1 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:51.144756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:51.145398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:51.145675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:51.145954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:51.146660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:51.150418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:51.150418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:51.161666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 648/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 22 10:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:51.167668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:51.274629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f0b5-a3b2-44d6-9740-1e4de25d4e04 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:51.291102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:51.310526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:51.311217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:51.311499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:51.311817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:51.319333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:51.320787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 22 10:49:51.332484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:51.334412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: ERROR keystone.server.flask.application Aug 22 10:49:51.335399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 637/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 22 10:49:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:51.340342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:51.392827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:51.443621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f94e79-1dae-4ada-946c-c5414e8bdee8 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:51.476391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fab4b6-c01f-48c5-9991-01164697e535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:51.477189 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fab4b6-c01f-48c5-9991-01164697e535 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:51.477467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fab4b6-c01f-48c5-9991-01164697e535 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:51.477734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fab4b6-c01f-48c5-9991-01164697e535 None admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:51.478438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fab4b6-c01f-48c5-9991-01164697e535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:51.478800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fab4b6-c01f-48c5-9991-01164697e535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:51.480213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fab4b6-c01f-48c5-9991-01164697e535 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:51.492391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 649/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 22 10:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:51.498503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:51.596259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc0b18-96f0-4e1d-9940-8257d12eae24 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:51.629875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42366cc5-4125-4620-94f3-54bc1c21b8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:51.630504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42366cc5-4125-4620-94f3-54bc1c21b8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:51.630786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42366cc5-4125-4620-94f3-54bc1c21b8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:51.631065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42366cc5-4125-4620-94f3-54bc1c21b8c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:51.631863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42366cc5-4125-4620-94f3-54bc1c21b8c7 None admin] RBAC: Adding query filter params (role.id=732ca52314ca470d86d12ac1b06cd16a, scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c, user.id=f3d2213b387f4938ae20eeb16c9352de) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:51.632174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42366cc5-4125-4620-94f3-54bc1c21b8c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:51.633628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42366cc5-4125-4620-94f3-54bc1c21b8c7 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:51.639415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 638/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 22 10:49:51 2022] GET /identity/v3/role_assignments?user.id=f3d2213b387f4938ae20eeb16c9352de&scope.project.id=e46a0cfe4c2b4edf9d5e27161f97e08c&role.id=732ca52314ca470d86d12ac1b06cd16a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:53.554862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e902a66-418b-4d15-87e6-b16bad49ac98 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:53.555393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e902a66-418b-4d15-87e6-b16bad49ac98 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:53.555393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e902a66-418b-4d15-87e6-b16bad49ac98 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:53.556788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 650/1288] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:53.562042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592d49b-7378-4403-abcf-a6d5c899b363 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:53.563689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592d49b-7378-4403-abcf-a6d5c899b363 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:53.563689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592d49b-7378-4403-abcf-a6d5c899b363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:53.580345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f592d49b-7378-4403-abcf-a6d5c899b363 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:53.581567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f592d49b-7378-4403-abcf-a6d5c899b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:53.610311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:53.647005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 639/1289] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:53.917022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e902a66-418b-4d15-87e6-b16bad49ac98 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:54.015809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e902a66-418b-4d15-87e6-b16bad49ac98 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:54.026783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:54.048895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:54.049532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:54.049795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:54.050130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:54.050776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:54.052148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:54.060539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:54.061727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:54.062804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 651/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:49:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:54.068250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592d49b-7378-4403-abcf-a6d5c899b363 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:54.169055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592d49b-7378-4403-abcf-a6d5c899b363 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:54.202998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ca89c-a73b-469d-a7c8-54e11995f387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:54.203608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ca89c-a73b-469d-a7c8-54e11995f387 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:54.203884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ca89c-a73b-469d-a7c8-54e11995f387 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:54.204154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ca89c-a73b-469d-a7c8-54e11995f387 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:54.204776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ca89c-a73b-469d-a7c8-54e11995f387 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:54.205073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ca89c-a73b-469d-a7c8-54e11995f387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:54.207020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ca89c-a73b-469d-a7c8-54e11995f387 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:54.211721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 640/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 22 10:49:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 10:49:54.217760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:54.326255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d1ac-d568-4d3f-b69a-ef40609c96d5 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:54.358999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274128-f091-4523-b4f5-137f0209897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:54.359596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274128-f091-4523-b4f5-137f0209897f None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:54.359877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274128-f091-4523-b4f5-137f0209897f None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:54.360149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274128-f091-4523-b4f5-137f0209897f None admin] PATH_INFO: `/v3/services` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:54.360772 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274128-f091-4523-b4f5-137f0209897f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:54.361071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274128-f091-4523-b4f5-137f0209897f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:54.362459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274128-f091-4523-b4f5-137f0209897f None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:54.367749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 652/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 22 10:49:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:56.254905 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5798bae-5874-4f16-a59c-f87ab67e4105 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:56.255449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5798bae-5874-4f16-a59c-f87ab67e4105 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:56.255449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5798bae-5874-4f16-a59c-f87ab67e4105 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:56.256813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 641/1293] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:56.262213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa713-734c-4e4a-978b-07b7f7d0c2e8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:56.262635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa713-734c-4e4a-978b-07b7f7d0c2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:56.262802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfa713-734c-4e4a-978b-07b7f7d0c2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:56.280759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6bcfa713-734c-4e4a-978b-07b7f7d0c2e8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:56.281982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6bcfa713-734c-4e4a-978b-07b7f7d0c2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:56.312153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:56.348692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 653/1294] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:56.613567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5798bae-5874-4f16-a59c-f87ab67e4105 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:56.644941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:56.712953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5798bae-5874-4f16-a59c-f87ab67e4105 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:56.747576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e633a-32b0-4a35-b406-36d2daaaef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:56.748183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e633a-32b0-4a35-b406-36d2daaaef6a None admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:56.748464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e633a-32b0-4a35-b406-36d2daaaef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:56.748734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e633a-32b0-4a35-b406-36d2daaaef6a None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:56.749364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e633a-32b0-4a35-b406-36d2daaaef6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:56.750813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e633a-32b0-4a35-b406-36d2daaaef6a None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:56.826610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 642/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 22 10:49:56 2022] POST /identity/v3/services => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:49:58.716829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5756-3720-4b82-99f9-30beddcfca5f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:58.717382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5756-3720-4b82-99f9-30beddcfca5f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:58.717382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5756-3720-4b82-99f9-30beddcfca5f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:58.718757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 654/1296] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:49:58.724446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a73efc-8f80-4baa-9e73-bd44c17d5d55 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:58.724860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a73efc-8f80-4baa-9e73-bd44c17d5d55 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:58.724981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a73efc-8f80-4baa-9e73-bd44c17d5d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:58.743229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f8a73efc-8f80-4baa-9e73-bd44c17d5d55 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:49:58.744389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f8a73efc-8f80-4baa-9e73-bd44c17d5d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:49:58.815706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 643/1297] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:49:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 22 10:49:59.079525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a5756-3720-4b82-99f9-30beddcfca5f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:59.181520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a5756-3720-4b82-99f9-30beddcfca5f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:59.215935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:59.216554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:59.216834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:59.217105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:59.217809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:59.219225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:49:59.229278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:49:59.233872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:49:59.234952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 655/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:49:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:49:59.236633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a73efc-8f80-4baa-9e73-bd44c17d5d55 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:59.256147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:59.336061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a73efc-8f80-4baa-9e73-bd44c17d5d55 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:59.368765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:59.369376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:59.369656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:59.369929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:59.370582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:59.370884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:59.372251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:59.377205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 644/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 22 10:49:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:59.383980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:59.414728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:59.480642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f58d13-4afd-4b3b-bac7-657be9a84444 None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:59.513466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c16eba-ee76-4282-906b-de0ab1dea206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:59.514061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16eba-ee76-4282-906b-de0ab1dea206 None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:59.514364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16eba-ee76-4282-906b-de0ab1dea206 None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:59.514695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16eba-ee76-4282-906b-de0ab1dea206 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:59.515382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c16eba-ee76-4282-906b-de0ab1dea206 None admin] RBAC: Adding query filter params (interface=public, service_id=0a3db81269d24e458488904202b439a4, region_id=RegionOne) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:49:59.515685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c16eba-ee76-4282-906b-de0ab1dea206 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:59.517052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c16eba-ee76-4282-906b-de0ab1dea206 None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:59.522683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 656/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 22 10:49:59 2022] GET /identity/v3/endpoints?service_id=0a3db81269d24e458488904202b439a4&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:49:59.528911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:49:59.609937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:49:59.627425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76632ad6-5737-41d6-b5ef-5849c28a86b8 None admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:49:59.660240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9374bca-a7e4-4f98-9d67-dcb971ca4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:49:59.661083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374bca-a7e4-4f98-9d67-dcb971ca4b86 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:49:59.661532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374bca-a7e4-4f98-9d67-dcb971ca4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:49:59.661855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374bca-a7e4-4f98-9d67-dcb971ca4b86 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:49:59.662857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9374bca-a7e4-4f98-9d67-dcb971ca4b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:49:59.664235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9374bca-a7e4-4f98-9d67-dcb971ca4b86 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:49:59.671252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 645/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 22 10:49:59 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 10:50:01.600832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baac4e0-64df-4ee0-8487-51fc10581214 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:50:01.601363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baac4e0-64df-4ee0-8487-51fc10581214 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:50:01.601363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baac4e0-64df-4ee0-8487-51fc10581214 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:50:01.602762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 657/1302] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:50:01.607911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916831d7-6bae-4fc3-a44b-5d00c197df08 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:50:01.608371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916831d7-6bae-4fc3-a44b-5d00c197df08 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:50:01.608464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916831d7-6bae-4fc3-a44b-5d00c197df08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:50:01.628535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-916831d7-6bae-4fc3-a44b-5d00c197df08 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:50:01.629699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-916831d7-6bae-4fc3-a44b-5d00c197df08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:50:01.695265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 646/1303] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:50:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 22 10:50:01.966447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baac4e0-64df-4ee0-8487-51fc10581214 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:50:02.061921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:50:02.069281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baac4e0-64df-4ee0-8487-51fc10581214 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:50:02.101388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:50:02.102000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:50:02.102414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:50:02.102694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:50:02.103263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:50:02.104598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 22 10:50:02.117870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 22 10:50:02.119126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.get_or_create( Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application with Lock( Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self._enter() Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application return self.creator() Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application created_value = creator( Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: ERROR keystone.server.flask.application Aug 22 10:50:02.120192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 658/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 22 10:50:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 22 10:50:02.121412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916831d7-6bae-4fc3-a44b-5d00c197df08 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:50:02.202124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:50:02.224335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916831d7-6bae-4fc3-a44b-5d00c197df08 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:50:02.258180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d88022-d0cf-4369-84d2-a5b21256f8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:50:02.258839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88022-d0cf-4369-84d2-a5b21256f8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:50:02.259115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88022-d0cf-4369-84d2-a5b21256f8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:50:02.259382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d88022-d0cf-4369-84d2-a5b21256f8b3 None admin] PATH_INFO: `/v3/services` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:50:02.260164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d88022-d0cf-4369-84d2-a5b21256f8b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 10:50:02.260476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d88022-d0cf-4369-84d2-a5b21256f8b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:50:02.262189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d88022-d0cf-4369-84d2-a5b21256f8b3 None admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:50:02.267664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 647/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 22 10:50:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 10:50:02.273692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:50:02.371583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c11eb-4191-488b-9b96-c95b9a1d3f1d None admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:50:02.403607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f4787-ca50-486e-936f-d4658e01fa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:50:02.404199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4787-ca50-486e-936f-d4658e01fa6c None admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:50:02.404473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4787-ca50-486e-936f-d4658e01fa6c None admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:50:02.404741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4787-ca50-486e-936f-d4658e01fa6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:50:02.405393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f4787-ca50-486e-936f-d4658e01fa6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:50:02.406781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f4787-ca50-486e-936f-d4658e01fa6c None admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:50:02.501268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 659/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 10:50:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 10:50:12.264614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=30 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:50:12.496562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:08.402201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d9c7-4750-48cd-9020-81e0508650bc None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:08.403149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d9c7-4750-48cd-9020-81e0508650bc None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:08.403573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d9c7-4750-48cd-9020-81e0508650bc None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:08.405077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 648/1307] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:08.411450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4aa4c4-9b86-452c-86f2-f03b0f9aec59 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:08.411924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4aa4c4-9b86-452c-86f2-f03b0f9aec59 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:08.412022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4aa4c4-9b86-452c-86f2-f03b0f9aec59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:08.550545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-3f4aa4c4-9b86-452c-86f2-f03b0f9aec59 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:08.552207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3f4aa4c4-9b86-452c-86f2-f03b0f9aec59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:08.683281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 660/1308] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:08.998515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6078a3-96f5-45cd-b647-e5bc5f7d7377 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:08.998909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6078a3-96f5-45cd-b647-e5bc5f7d7377 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:08.999283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6078a3-96f5-45cd-b647-e5bc5f7d7377 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:09.001228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 649/1309] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 22 10:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:09.009459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e70a-17ca-4136-8a48-9c43aa3c22dc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:09.009770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e70a-17ca-4136-8a48-9c43aa3c22dc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:09.009998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02e70a-17ca-4136-8a48-9c43aa3c22dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:09.154555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2e02e70a-17ca-4136-8a48-9c43aa3c22dc None None] MFA Rules not processed for user `edd00e00b48641c8bcaacaaa7c2dd149`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:09.160320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2e02e70a-17ca-4136-8a48-9c43aa3c22dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:09.197291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:09.273147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6078a3-96f5-45cd-b647-e5bc5f7d7377 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:09.274956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fb6078a3-96f5-45cd-b647-e5bc5f7d7377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:09.279067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 661/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 22 10:56:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:09.413464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6078a3-96f5-45cd-b647-e5bc5f7d7377 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:09.507130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a983ee9-2ac6-4513-8da2-c3f1351af331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:09.507774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a983ee9-2ac6-4513-8da2-c3f1351af331 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:09.508075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a983ee9-2ac6-4513-8da2-c3f1351af331 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:09.508491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a983ee9-2ac6-4513-8da2-c3f1351af331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:09.510131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7a983ee9-2ac6-4513-8da2-c3f1351af331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:09.580262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:09.650536 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a983ee9-2ac6-4513-8da2-c3f1351af331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:09.652410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a983ee9-2ac6-4513-8da2-c3f1351af331 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:09.676609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 650/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:56:11.939578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d2a2-4991-4861-9876-79d4e494074d None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:11.940160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d2a2-4991-4861-9876-79d4e494074d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:11.940160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d2a2-4991-4861-9876-79d4e494074d None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:11.941512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 662/1312] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:11.947494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16d2f6-ec91-4dee-87ae-0c04c58e0619 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:11.948020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16d2f6-ec91-4dee-87ae-0c04c58e0619 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:11.948143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16d2f6-ec91-4dee-87ae-0c04c58e0619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:12.030636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6d16d2f6-ec91-4dee-87ae-0c04c58e0619 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:12.032741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6d16d2f6-ec91-4dee-87ae-0c04c58e0619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:12.135624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 651/1313] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:12.429775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d2a2-4991-4861-9876-79d4e494074d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:12.431377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d592d2a2-4991-4861-9876-79d4e494074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:12.533318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d2a2-4991-4861-9876-79d4e494074d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:12.601682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:12.604251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a8c2d-9285-473f-b9cc-59a3ea7ec911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:12.604843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a8c2d-9285-473f-b9cc-59a3ea7ec911 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:12.605119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a8c2d-9285-473f-b9cc-59a3ea7ec911 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:12.605386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a8c2d-9285-473f-b9cc-59a3ea7ec911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:12.606995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bc7a8c2d-9285-473f-b9cc-59a3ea7ec911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:12.714412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a8c2d-9285-473f-b9cc-59a3ea7ec911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:12.715849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a8c2d-9285-473f-b9cc-59a3ea7ec911 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:12.733894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 663/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:15.148612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c24cb-b0d4-46ae-9f1b-37f99e4f0f02 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:15.149147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c24cb-b0d4-46ae-9f1b-37f99e4f0f02 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:15.149147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c24cb-b0d4-46ae-9f1b-37f99e4f0f02 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:15.150446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 652/1315] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:15.155877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c770b0-a1b9-4698-850f-90bf5d44c64f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:15.156272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c770b0-a1b9-4698-850f-90bf5d44c64f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:15.156426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c770b0-a1b9-4698-850f-90bf5d44c64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:15.221498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b0c770b0-a1b9-4698-850f-90bf5d44c64f None None] MFA Rules not processed for user `61475891ae6e4dce9c537af85750743e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:15.222742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b0c770b0-a1b9-4698-850f-90bf5d44c64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:15.274084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:15.318040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 664/1316] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:15.602188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c24cb-b0d4-46ae-9f1b-37f99e4f0f02 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:15.603731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bd4c24cb-b0d4-46ae-9f1b-37f99e4f0f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:15.613821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:15.706448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c24cb-b0d4-46ae-9f1b-37f99e4f0f02 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:15.778076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274547cd-a1be-497f-b1ee-f2ee67e7a52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:15.778711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274547cd-a1be-497f-b1ee-f2ee67e7a52d service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:15.778996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274547cd-a1be-497f-b1ee-f2ee67e7a52d service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:15.779266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274547cd-a1be-497f-b1ee-f2ee67e7a52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:15.780847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-274547cd-a1be-497f-b1ee-f2ee67e7a52d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:15.841348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:15.889459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274547cd-a1be-497f-b1ee-f2ee67e7a52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:15.890930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274547cd-a1be-497f-b1ee-f2ee67e7a52d service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:15.909213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 653/1317] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:18.598555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cafc1-1dfb-40bf-b96a-21745d2bcaff None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:18.599058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cafc1-1dfb-40bf-b96a-21745d2bcaff None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:18.599058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cafc1-1dfb-40bf-b96a-21745d2bcaff None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:18.600354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 665/1318] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:18.605427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef9c1b-5424-4010-a8e5-d38a7104a58a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:18.605827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef9c1b-5424-4010-a8e5-d38a7104a58a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:18.605982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef9c1b-5424-4010-a8e5-d38a7104a58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:18.671088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-abef9c1b-5424-4010-a8e5-d38a7104a58a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:18.672285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-abef9c1b-5424-4010-a8e5-d38a7104a58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:18.766868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 654/1319] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:19.057053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cafc1-1dfb-40bf-b96a-21745d2bcaff None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:19.058631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-441cafc1-1dfb-40bf-b96a-21745d2bcaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:19.157885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cafc1-1dfb-40bf-b96a-21745d2bcaff None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:19.228235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c71df7-ca84-4279-98cc-7adb77624554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:19.228827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71df7-ca84-4279-98cc-7adb77624554 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:19.229102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71df7-ca84-4279-98cc-7adb77624554 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:19.229369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c71df7-ca84-4279-98cc-7adb77624554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:19.230954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-35c71df7-ca84-4279-98cc-7adb77624554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:19.238242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:19.345492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c71df7-ca84-4279-98cc-7adb77624554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:19.347057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c71df7-ca84-4279-98cc-7adb77624554 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:19.365007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 666/1320] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:22.355050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c7b5-6cb9-4f0f-a0f6-0a174c83132b None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:22.355547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c7b5-6cb9-4f0f-a0f6-0a174c83132b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:22.355645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1c7b5-6cb9-4f0f-a0f6-0a174c83132b None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:22.357102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 655/1321] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:22.362545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6b9d9-ffe5-45ec-8124-0b3308509a4d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:22.363004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6b9d9-ffe5-45ec-8124-0b3308509a4d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:22.363114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6b9d9-ffe5-45ec-8124-0b3308509a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:22.430613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-96e6b9d9-ffe5-45ec-8124-0b3308509a4d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:22.431827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-96e6b9d9-ffe5-45ec-8124-0b3308509a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:22.489589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:22.528331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 667/1322] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:22.836216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ae29-dbe5-4943-8ebc-9e6e4fc31e28 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:22.836497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ae29-dbe5-4943-8ebc-9e6e4fc31e28 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:22.836773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4ae29-dbe5-4943-8ebc-9e6e4fc31e28 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:22.838196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 656/1323] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 22 10:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:22.845197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f55416-fd28-4f73-ae2d-2f9efbb86596 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:22.845557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f55416-fd28-4f73-ae2d-2f9efbb86596 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:22.850026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f55416-fd28-4f73-ae2d-2f9efbb86596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:22.930417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f8f55416-fd28-4f73-ae2d-2f9efbb86596 None None] MFA Rules not processed for user `edd00e00b48641c8bcaacaaa7c2dd149`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:22.931652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f8f55416-fd28-4f73-ae2d-2f9efbb86596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:23.008841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 668/1324] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 22 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:23.018478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4ae29-dbe5-4943-8ebc-9e6e4fc31e28 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:23.020048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-49e4ae29-dbe5-4943-8ebc-9e6e4fc31e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:23.043062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:23.131547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4ae29-dbe5-4943-8ebc-9e6e4fc31e28 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:23.206740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a8bd3-84ff-4765-81b3-2c8f1192a838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:23.207349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a8bd3-84ff-4765-81b3-2c8f1192a838 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:23.207627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a8bd3-84ff-4765-81b3-2c8f1192a838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:23.207895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a8bd3-84ff-4765-81b3-2c8f1192a838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:23.209748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d88a8bd3-84ff-4765-81b3-2c8f1192a838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:23.281794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:23.326116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a8bd3-84ff-4765-81b3-2c8f1192a838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:23.329262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a8bd3-84ff-4765-81b3-2c8f1192a838 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:23.361618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 657/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:25.598521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c53c57-ce84-4aeb-b2f9-960c393d1fdd None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:25.599045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c53c57-ce84-4aeb-b2f9-960c393d1fdd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:25.599155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c53c57-ce84-4aeb-b2f9-960c393d1fdd None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:25.600462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 669/1326] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:25.605769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee820-4c91-40a3-b9b7-aad0c4ac2a87 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:25.606179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee820-4c91-40a3-b9b7-aad0c4ac2a87 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:25.606368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aee820-4c91-40a3-b9b7-aad0c4ac2a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:25.672981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-67aee820-4c91-40a3-b9b7-aad0c4ac2a87 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:25.674196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-67aee820-4c91-40a3-b9b7-aad0c4ac2a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:25.783807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 658/1327] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:26.068864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c53c57-ce84-4aeb-b2f9-960c393d1fdd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:26.070520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-36c53c57-ce84-4aeb-b2f9-960c393d1fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:26.131095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:26.173737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c53c57-ce84-4aeb-b2f9-960c393d1fdd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:26.249047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e969c-0374-4703-9234-115781293d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:26.249654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e969c-0374-4703-9234-115781293d4a service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:26.249933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e969c-0374-4703-9234-115781293d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:26.250204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e969c-0374-4703-9234-115781293d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:26.251941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ed6e969c-0374-4703-9234-115781293d4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:26.360975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:26.363570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e969c-0374-4703-9234-115781293d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:26.365046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e969c-0374-4703-9234-115781293d4a service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:26.383498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 670/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:28.517812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc8d0-fb7f-4cb9-9917-89930b081a2d None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:28.518350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc8d0-fb7f-4cb9-9917-89930b081a2d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:28.518458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc8d0-fb7f-4cb9-9917-89930b081a2d None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:28.519737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 659/1329] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:28.525028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90e037-0621-4706-a4c9-61b529dcda84 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:28.525437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90e037-0621-4706-a4c9-61b529dcda84 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:28.525609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90e037-0621-4706-a4c9-61b529dcda84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:28.590884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-cc90e037-0621-4706-a4c9-61b529dcda84 None None] MFA Rules not processed for user `61475891ae6e4dce9c537af85750743e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:28.592073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cc90e037-0621-4706-a4c9-61b529dcda84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:28.686996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 671/1330] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:28.973879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc8d0-fb7f-4cb9-9917-89930b081a2d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:28.975563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0a7dc8d0-fb7f-4cb9-9917-89930b081a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:29.007911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:29.084455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc8d0-fb7f-4cb9-9917-89930b081a2d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:29.163124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117510cf-d0bb-4304-9f13-a6dbae5a9fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:29.163749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117510cf-d0bb-4304-9f13-a6dbae5a9fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:29.164035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117510cf-d0bb-4304-9f13-a6dbae5a9fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:29.164310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117510cf-d0bb-4304-9f13-a6dbae5a9fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:29.165970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-117510cf-d0bb-4304-9f13-a6dbae5a9fe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:29.246970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:29.278791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117510cf-d0bb-4304-9f13-a6dbae5a9fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:29.280236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117510cf-d0bb-4304-9f13-a6dbae5a9fe7 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:29.299383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 660/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:31.703058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b89f7-e2c0-4f8f-be5f-cd092519e0d7 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:31.703767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b89f7-e2c0-4f8f-be5f-cd092519e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:31.703767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b89f7-e2c0-4f8f-be5f-cd092519e0d7 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:31.705045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 672/1332] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:31.710387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df587f1-25a6-4664-9f65-921f83393967 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:31.710815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df587f1-25a6-4664-9f65-921f83393967 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:31.710936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df587f1-25a6-4664-9f65-921f83393967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:31.775324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3df587f1-25a6-4664-9f65-921f83393967 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:31.776507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3df587f1-25a6-4664-9f65-921f83393967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:31.872908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 661/1333] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:32.154403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b89f7-e2c0-4f8f-be5f-cd092519e0d7 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:32.155935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3a0b89f7-e2c0-4f8f-be5f-cd092519e0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:32.223600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:32.256629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b89f7-e2c0-4f8f-be5f-cd092519e0d7 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:32.328213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8c198-1f78-4818-8fb3-24d216745f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:32.328970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8c198-1f78-4818-8fb3-24d216745f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:32.329246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8c198-1f78-4818-8fb3-24d216745f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:32.329516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8c198-1f78-4818-8fb3-24d216745f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:32.331120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-54b8c198-1f78-4818-8fb3-24d216745f3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:32.439077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8c198-1f78-4818-8fb3-24d216745f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:32.440422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8c198-1f78-4818-8fb3-24d216745f3a service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:32.450146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:32.458658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 673/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:34.413016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96732370-f3d4-4e63-ae2c-cce25b33d707 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:34.413554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96732370-f3d4-4e63-ae2c-cce25b33d707 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:34.413554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96732370-f3d4-4e63-ae2c-cce25b33d707 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:34.414959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 662/1335] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:34.420066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e1265-9e00-4e24-a970-77162613607d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:34.420501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e1265-9e00-4e24-a970-77162613607d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:34.420618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e1265-9e00-4e24-a970-77162613607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:34.484546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f60e1265-9e00-4e24-a970-77162613607d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:34.485748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f60e1265-9e00-4e24-a970-77162613607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:34.580735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 674/1336] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:34.877895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96732370-f3d4-4e63-ae2c-cce25b33d707 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:34.879462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-96732370-f3d4-4e63-ae2c-cce25b33d707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:34.917825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:34.981372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96732370-f3d4-4e63-ae2c-cce25b33d707 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:35.053234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013b47c-6f2f-414a-ad63-6e7f7c3c073c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:35.053841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013b47c-6f2f-414a-ad63-6e7f7c3c073c service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:35.054119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013b47c-6f2f-414a-ad63-6e7f7c3c073c service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:35.054423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013b47c-6f2f-414a-ad63-6e7f7c3c073c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:35.056002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2013b47c-6f2f-414a-ad63-6e7f7c3c073c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:35.138149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:35.164111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013b47c-6f2f-414a-ad63-6e7f7c3c073c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:35.165545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013b47c-6f2f-414a-ad63-6e7f7c3c073c service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:35.183851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 663/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:37.633185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627aa0c-8333-43b6-83fc-214fb57185a6 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:37.633777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627aa0c-8333-43b6-83fc-214fb57185a6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:37.633777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627aa0c-8333-43b6-83fc-214fb57185a6 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:37.635114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 675/1338] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:37.640376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900cd57-c004-4cc7-aeb7-05323e83bd0d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:37.641488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900cd57-c004-4cc7-aeb7-05323e83bd0d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:37.641579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900cd57-c004-4cc7-aeb7-05323e83bd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:37.706076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c900cd57-c004-4cc7-aeb7-05323e83bd0d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:37.707316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c900cd57-c004-4cc7-aeb7-05323e83bd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:37.801623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 664/1339] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:38.083372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627aa0c-8333-43b6-83fc-214fb57185a6 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:38.084919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2627aa0c-8333-43b6-83fc-214fb57185a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:38.159384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:38.186055 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627aa0c-8333-43b6-83fc-214fb57185a6 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:38.256965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d6f082-9723-4b54-b4e8-07657b95e119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:38.257562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6f082-9723-4b54-b4e8-07657b95e119 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:38.257837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6f082-9723-4b54-b4e8-07657b95e119 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:38.258105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6f082-9723-4b54-b4e8-07657b95e119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:38.259699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-46d6f082-9723-4b54-b4e8-07657b95e119 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:38.366410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6f082-9723-4b54-b4e8-07657b95e119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:38.367842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d6f082-9723-4b54-b4e8-07657b95e119 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:38.385804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 676/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:42.487021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9175-71b6-49db-81a8-8d155e680103 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:42.487573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9175-71b6-49db-81a8-8d155e680103 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:42.487573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9175-71b6-49db-81a8-8d155e680103 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:42.488914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 665/1341] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:42.494330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9485c-0f73-4862-aa3d-d557291ccd6f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:42.494732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9485c-0f73-4862-aa3d-d557291ccd6f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:42.495035 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9485c-0f73-4862-aa3d-d557291ccd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:42.501734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:42.566257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-24f9485c-0f73-4862-aa3d-d557291ccd6f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:42.567489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-24f9485c-0f73-4862-aa3d-d557291ccd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:42.665612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 677/1342] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:42.951042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a9175-71b6-49db-81a8-8d155e680103 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:42.952575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-361a9175-71b6-49db-81a8-8d155e680103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:43.003250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:43.058966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a9175-71b6-49db-81a8-8d155e680103 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:43.139853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b656618-7a10-460e-b694-5fbab65bb8b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:43.140495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b656618-7a10-460e-b694-5fbab65bb8b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:43.140768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b656618-7a10-460e-b694-5fbab65bb8b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:43.141041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b656618-7a10-460e-b694-5fbab65bb8b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:43.142644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8b656618-7a10-460e-b694-5fbab65bb8b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:43.235672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:43.252327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b656618-7a10-460e-b694-5fbab65bb8b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:43.253760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b656618-7a10-460e-b694-5fbab65bb8b9 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:43.271770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 666/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:46.615208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9c7dc-2a10-43d1-b95a-9104c33b5ec1 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:46.615737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9c7dc-2a10-43d1-b95a-9104c33b5ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:46.615737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9c7dc-2a10-43d1-b95a-9104c33b5ec1 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:46.617099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 678/1344] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:46.622668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dd64-938b-4d3d-b8ab-70191ff0ab0a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:46.623065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dd64-938b-4d3d-b8ab-70191ff0ab0a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:46.623209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99dd64-938b-4d3d-b8ab-70191ff0ab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:46.691369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6a99dd64-938b-4d3d-b8ab-70191ff0ab0a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:46.692587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6a99dd64-938b-4d3d-b8ab-70191ff0ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:46.791265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 667/1345] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:47.076354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9c7dc-2a10-43d1-b95a-9104c33b5ec1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:47.078551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5dc9c7dc-2a10-43d1-b95a-9104c33b5ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:47.168952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:47.185316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9c7dc-2a10-43d1-b95a-9104c33b5ec1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:47.259393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbca4aa-dfb3-4822-9fe7-b9944f5fc013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:47.259999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbca4aa-dfb3-4822-9fe7-b9944f5fc013 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:47.260278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbca4aa-dfb3-4822-9fe7-b9944f5fc013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:47.260547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbca4aa-dfb3-4822-9fe7-b9944f5fc013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:47.262144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3dbca4aa-dfb3-4822-9fe7-b9944f5fc013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:47.398913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbca4aa-dfb3-4822-9fe7-b9944f5fc013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:47.400333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbca4aa-dfb3-4822-9fe7-b9944f5fc013 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:47.420915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 679/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:51.752079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174de2b3-637c-485f-9420-cd9f6d71a82c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:51.752647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174de2b3-637c-485f-9420-cd9f6d71a82c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:51.752647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174de2b3-637c-485f-9420-cd9f6d71a82c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:51.753975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 668/1347] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:51.759679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2099-5de1-458d-977c-aefe9f413223 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:51.760229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2099-5de1-458d-977c-aefe9f413223 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:51.760229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f2099-5de1-458d-977c-aefe9f413223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:51.776805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:51.834260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-361f2099-5de1-458d-977c-aefe9f413223 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:51.835508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-361f2099-5de1-458d-977c-aefe9f413223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:51.939064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 680/1348] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:56:52.223641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174de2b3-637c-485f-9420-cd9f6d71a82c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:56:52.225160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-174de2b3-637c-485f-9420-cd9f6d71a82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:52.282791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:52.329251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174de2b3-637c-485f-9420-cd9f6d71a82c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:56:52.406799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af633934-889c-4132-8c6b-d8ffb8dd49fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:56:52.407416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af633934-889c-4132-8c6b-d8ffb8dd49fc service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:52.407696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af633934-889c-4132-8c6b-d8ffb8dd49fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:52.407970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af633934-889c-4132-8c6b-d8ffb8dd49fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:52.409568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-af633934-889c-4132-8c6b-d8ffb8dd49fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:52.517268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:56:52.520613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af633934-889c-4132-8c6b-d8ffb8dd49fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:56:52.522069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af633934-889c-4132-8c6b-d8ffb8dd49fc service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:56:52.540653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 669/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:56:57.534677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c0e5-9813-47b9-84d3-ee0896289f2f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:57.535160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c0e5-9813-47b9-84d3-ee0896289f2f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:57.535160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450c0e5-9813-47b9-84d3-ee0896289f2f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:57.548065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 681/1350] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 10:56:57 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:56:57.554607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647d94-e657-457c-bc37-f8146f1d878d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:56:57.556112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647d94-e657-457c-bc37-f8146f1d878d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:56:57.556667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647d94-e657-457c-bc37-f8146f1d878d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:56:57.678449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f3647d94-e657-457c-bc37-f8146f1d878d None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:56:57.679715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f3647d94-e657-457c-bc37-f8146f1d878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:56:57.853982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 670/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:01.937129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=59 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:02.083303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c59f7-e2cc-444a-81e1-dc3c541ab139 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:02.083731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c59f7-e2cc-444a-81e1-dc3c541ab139 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:02.083851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c59f7-e2cc-444a-81e1-dc3c541ab139 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:02.085411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 682/1352] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 10:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:02.090321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966b5a0-a17e-47b7-907d-450c337f790d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:02.090822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966b5a0-a17e-47b7-907d-450c337f790d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:02.090919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966b5a0-a17e-47b7-907d-450c337f790d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:02.147002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:02.170150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2966b5a0-a17e-47b7-907d-450c337f790d None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:02.172126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2966b5a0-a17e-47b7-907d-450c337f790d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:02.282472 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 671/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:03.673684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118bfd6-41a4-488e-ad82-a0933e95868a None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:03.674263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118bfd6-41a4-488e-ad82-a0933e95868a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:03.674263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118bfd6-41a4-488e-ad82-a0933e95868a None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:03.675617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 683/1354] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 10:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:03.680746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278135c-5c30-4b44-88ac-21ebb4ef2f32 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:03.681165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278135c-5c30-4b44-88ac-21ebb4ef2f32 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:03.681306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278135c-5c30-4b44-88ac-21ebb4ef2f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:03.758171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c278135c-5c30-4b44-88ac-21ebb4ef2f32 None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:03.759411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c278135c-5c30-4b44-88ac-21ebb4ef2f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:03.803778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:03.873093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 672/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 10:57:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:57:07.100343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca51ef0-3b6b-4cb8-b9f3-a98107cee496 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:07.100901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca51ef0-3b6b-4cb8-b9f3-a98107cee496 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:07.100901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca51ef0-3b6b-4cb8-b9f3-a98107cee496 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:07.102209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 684/1356] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:07.107473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944be22-d49e-4d4d-9574-f672227ec736 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:07.108922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944be22-d49e-4d4d-9574-f672227ec736 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:07.109188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944be22-d49e-4d4d-9574-f672227ec736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:07.180870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5944be22-d49e-4d4d-9574-f672227ec736 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:07.182110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5944be22-d49e-4d4d-9574-f672227ec736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:07.318516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 673/1357] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:07.619024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca7c5-9ffc-48a8-8556-d289a03f48e8 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:07.619581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca7c5-9ffc-48a8-8556-d289a03f48e8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:07.620324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca7c5-9ffc-48a8-8556-d289a03f48e8 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:07.621926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 685/1358] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 22 10:57:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:07.629229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031f31-8762-48cb-ab1e-c3ec3026cf36 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:07.629519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031f31-8762-48cb-ab1e-c3ec3026cf36 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:07.629789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc031f31-8762-48cb-ab1e-c3ec3026cf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:07.653239 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:07.717624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-bc031f31-8762-48cb-ab1e-c3ec3026cf36 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:07.719162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bc031f31-8762-48cb-ab1e-c3ec3026cf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:07.850139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 674/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 22 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:07.859042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ca7c5-9ffc-48a8-8556-d289a03f48e8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:07.860679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-892ca7c5-9ffc-48a8-8556-d289a03f48e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:08.041187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ca7c5-9ffc-48a8-8556-d289a03f48e8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:08.122987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:08.164437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26adf6-b78c-4dbd-853c-3f063b2e7631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:08.165212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26adf6-b78c-4dbd-853c-3f063b2e7631 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:08.165586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26adf6-b78c-4dbd-853c-3f063b2e7631 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:08.165969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26adf6-b78c-4dbd-853c-3f063b2e7631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:08.167721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cd26adf6-b78c-4dbd-853c-3f063b2e7631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:08.262417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26adf6-b78c-4dbd-853c-3f063b2e7631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:08.264920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd26adf6-b78c-4dbd-853c-3f063b2e7631 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:08.285303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 686/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:10.442763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f387cd-4469-4afd-a0e9-56d9e94462d9 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:10.443176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f387cd-4469-4afd-a0e9-56d9e94462d9 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:10.443309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f387cd-4469-4afd-a0e9-56d9e94462d9 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:10.444917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 675/1361] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:10.450261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b7eed-f1d4-46a3-a407-370e3d26fa22 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:10.450914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b7eed-f1d4-46a3-a407-370e3d26fa22 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:10.450914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b7eed-f1d4-46a3-a407-370e3d26fa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:10.519912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bd9b7eed-f1d4-46a3-a407-370e3d26fa22 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:10.521116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bd9b7eed-f1d4-46a3-a407-370e3d26fa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:10.560387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:10.634895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 687/1362] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:10.921383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f387cd-4469-4afd-a0e9-56d9e94462d9 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:10.922958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b1f387cd-4469-4afd-a0e9-56d9e94462d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:10.991176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:11.074429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f387cd-4469-4afd-a0e9-56d9e94462d9 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:11.197297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3327ca-83be-4aea-9088-9b1cb2599866 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:11.198077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3327ca-83be-4aea-9088-9b1cb2599866 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:11.198415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3327ca-83be-4aea-9088-9b1cb2599866 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:11.198691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3327ca-83be-4aea-9088-9b1cb2599866 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:11.200285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6b3327ca-83be-4aea-9088-9b1cb2599866 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:11.227509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:11.309902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3327ca-83be-4aea-9088-9b1cb2599866 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:11.312571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3327ca-83be-4aea-9088-9b1cb2599866 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:11.333221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 676/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:13.693559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751008-fc18-454e-965b-f634d1a92e30 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:13.694153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751008-fc18-454e-965b-f634d1a92e30 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:13.694153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751008-fc18-454e-965b-f634d1a92e30 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:13.695464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 688/1364] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:13.700817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662041d-b7b4-4428-8f68-2638f7146813 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:13.701823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662041d-b7b4-4428-8f68-2638f7146813 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:13.702168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662041d-b7b4-4428-8f68-2638f7146813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:13.770082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a662041d-b7b4-4428-8f68-2638f7146813 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:13.771316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a662041d-b7b4-4428-8f68-2638f7146813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:13.881989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:13.886315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 677/1365] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:14.186150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb4c31-0ea7-4c93-9fae-2b2e40a2222f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:14.186709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb4c31-0ea7-4c93-9fae-2b2e40a2222f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:14.187137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb4c31-0ea7-4c93-9fae-2b2e40a2222f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:14.189023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 689/1366] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 22 10:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:14.193904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972121-0f0e-4769-86b6-1a1f221bdcba None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:14.194335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972121-0f0e-4769-86b6-1a1f221bdcba None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:14.194705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30972121-0f0e-4769-86b6-1a1f221bdcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:14.279061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-30972121-0f0e-4769-86b6-1a1f221bdcba None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:14.281105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-30972121-0f0e-4769-86b6-1a1f221bdcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:14.406273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 678/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 22 10:57:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:14.414208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb4c31-0ea7-4c93-9fae-2b2e40a2222f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:14.415811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c2bb4c31-0ea7-4c93-9fae-2b2e40a2222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:14.586604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb4c31-0ea7-4c93-9fae-2b2e40a2222f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:14.610126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:14.714392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab7c5f-6e6b-429f-9161-6c736f7bed23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:14.715050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab7c5f-6e6b-429f-9161-6c736f7bed23 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:14.715290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab7c5f-6e6b-429f-9161-6c736f7bed23 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:14.715532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab7c5f-6e6b-429f-9161-6c736f7bed23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:14.717311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4aab7c5f-6e6b-429f-9161-6c736f7bed23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:14.817430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab7c5f-6e6b-429f-9161-6c736f7bed23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:14.819113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab7c5f-6e6b-429f-9161-6c736f7bed23 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:14.842647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 690/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:57:15.459319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0843-4322-4759-9fa2-0cb3735880a4 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:15.459908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0843-4322-4759-9fa2-0cb3735880a4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:15.459908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b0843-4322-4759-9fa2-0cb3735880a4 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:15.461387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 679/1369] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:15.466702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab803a-74e3-44cc-9fc2-970de09961e5 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:15.467065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab803a-74e3-44cc-9fc2-970de09961e5 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:15.467346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab803a-74e3-44cc-9fc2-970de09961e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:15.502266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:15.542657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-10ab803a-74e3-44cc-9fc2-970de09961e5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:15.544034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-10ab803a-74e3-44cc-9fc2-970de09961e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:15.658289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 691/1370] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:16.266304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c2aaa-49e7-4d88-96c3-0dabecdc9203 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:16.266538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c2aaa-49e7-4d88-96c3-0dabecdc9203 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:16.266813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c2aaa-49e7-4d88-96c3-0dabecdc9203 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:16.268303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 680/1371] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 22 10:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:16.273081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a4448-2b9b-4c44-aca9-4e0f0b4b08cd None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:16.273534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a4448-2b9b-4c44-aca9-4e0f0b4b08cd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:16.273724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a4448-2b9b-4c44-aca9-4e0f0b4b08cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:16.353529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f24a4448-2b9b-4c44-aca9-4e0f0b4b08cd None None] MFA Rules not processed for user `7cdb62860b6243bb84c31d1caff4bc77`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:16.354830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f24a4448-2b9b-4c44-aca9-4e0f0b4b08cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:16.405007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:16.436424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 692/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 22 10:57:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:16.444068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c2aaa-49e7-4d88-96c3-0dabecdc9203 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:16.445584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-579c2aaa-49e7-4d88-96c3-0dabecdc9203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:16.498356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:16.563698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c2aaa-49e7-4d88-96c3-0dabecdc9203 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:16.648563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0360d06-0b18-48d5-92ec-47725d16e86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:16.649173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360d06-0b18-48d5-92ec-47725d16e86b service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:16.649539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360d06-0b18-48d5-92ec-47725d16e86b service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:16.649716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0360d06-0b18-48d5-92ec-47725d16e86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:16.651337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f0360d06-0b18-48d5-92ec-47725d16e86b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:16.775983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:16.786981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0360d06-0b18-48d5-92ec-47725d16e86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:16.791280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0360d06-0b18-48d5-92ec-47725d16e86b service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:16.814585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 681/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:16.970511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04f26-2f70-442b-a378-bdd8f6223d32 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:16.970847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04f26-2f70-442b-a378-bdd8f6223d32 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:16.971426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b04f26-2f70-442b-a378-bdd8f6223d32 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:16.974709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 693/1374] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:16.977815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6d035-39c4-4b3d-b2d3-8de5d92c83ad None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:16.978108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6d035-39c4-4b3d-b2d3-8de5d92c83ad None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:16.978429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6d035-39c4-4b3d-b2d3-8de5d92c83ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:17.052443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-94d6d035-39c4-4b3d-b2d3-8de5d92c83ad None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:17.053695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-94d6d035-39c4-4b3d-b2d3-8de5d92c83ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:17.158908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 682/1375] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:17.464683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b04f26-2f70-442b-a378-bdd8f6223d32 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:17.466209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-61b04f26-2f70-442b-a378-bdd8f6223d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:17.616109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b04f26-2f70-442b-a378-bdd8f6223d32 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:17.667774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:17.729440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835917c3-cb9e-4766-917c-1edc8f6f4096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:17.730051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835917c3-cb9e-4766-917c-1edc8f6f4096 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:17.730368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835917c3-cb9e-4766-917c-1edc8f6f4096 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:17.730634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835917c3-cb9e-4766-917c-1edc8f6f4096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:17.732761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-835917c3-cb9e-4766-917c-1edc8f6f4096 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:17.822922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835917c3-cb9e-4766-917c-1edc8f6f4096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:17.824666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835917c3-cb9e-4766-917c-1edc8f6f4096 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:17.842878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 694/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:19.909818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde332a-d9e7-415b-a147-f1d10148d9a1 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:19.910342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde332a-d9e7-415b-a147-f1d10148d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:19.910441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde332a-d9e7-415b-a147-f1d10148d9a1 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:19.913339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 683/1377] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:19.920782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108ee6f-061d-43d6-82ed-4fa0a837ce8a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:19.921258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108ee6f-061d-43d6-82ed-4fa0a837ce8a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:19.921480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108ee6f-061d-43d6-82ed-4fa0a837ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:20.001132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f108ee6f-061d-43d6-82ed-4fa0a837ce8a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:20.002516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f108ee6f-061d-43d6-82ed-4fa0a837ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:20.010280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:20.111156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 695/1378] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:20.416801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde332a-d9e7-415b-a147-f1d10148d9a1 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:20.423051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fdde332a-d9e7-415b-a147-f1d10148d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:20.524122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:20.619377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde332a-d9e7-415b-a147-f1d10148d9a1 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:20.749897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d299c-e3b5-48c3-93d4-0b357be22927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:20.750566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d299c-e3b5-48c3-93d4-0b357be22927 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:20.750831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d299c-e3b5-48c3-93d4-0b357be22927 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:20.751101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d299c-e3b5-48c3-93d4-0b357be22927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:20.752848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e51d299c-e3b5-48c3-93d4-0b357be22927 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:20.775111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:20.849165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d299c-e3b5-48c3-93d4-0b357be22927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:20.851023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d299c-e3b5-48c3-93d4-0b357be22927 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:20.875412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 684/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:57:23.001269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b575f-14dc-4594-bde4-faa4f57eac97 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:23.001832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b575f-14dc-4594-bde4-faa4f57eac97 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:23.001832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b575f-14dc-4594-bde4-faa4f57eac97 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:23.003387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 696/1380] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:23.008470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6961e2-20dc-406f-9cd2-b5c7c6cf0f0b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:23.008875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6961e2-20dc-406f-9cd2-b5c7c6cf0f0b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:23.009052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6961e2-20dc-406f-9cd2-b5c7c6cf0f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:23.078638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3f6961e2-20dc-406f-9cd2-b5c7c6cf0f0b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:23.079761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3f6961e2-20dc-406f-9cd2-b5c7c6cf0f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:23.179666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:23.187311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 685/1381] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:23.489087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b575f-14dc-4594-bde4-faa4f57eac97 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:23.490656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a49b575f-14dc-4594-bde4-faa4f57eac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:23.654545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:23.665895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b575f-14dc-4594-bde4-faa4f57eac97 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:23.793748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38561e-dd32-480a-8023-caeecc8939d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:23.795146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38561e-dd32-480a-8023-caeecc8939d9 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:23.795432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38561e-dd32-480a-8023-caeecc8939d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:23.795709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38561e-dd32-480a-8023-caeecc8939d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:23.797528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9e38561e-dd32-480a-8023-caeecc8939d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:23.905364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38561e-dd32-480a-8023-caeecc8939d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:23.907209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38561e-dd32-480a-8023-caeecc8939d9 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:23.918675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:23.928072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 697/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:26.021410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6012d-14c2-42ea-bf7b-93e446694e8f None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:26.021959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6012d-14c2-42ea-bf7b-93e446694e8f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:26.022315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6012d-14c2-42ea-bf7b-93e446694e8f None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:26.024158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 686/1383] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:26.029681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6830a06-43d3-4260-bf74-30ea59c9d8bc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:26.030108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6830a06-43d3-4260-bf74-30ea59c9d8bc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:26.030357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6830a06-43d3-4260-bf74-30ea59c9d8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:26.104599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c6830a06-43d3-4260-bf74-30ea59c9d8bc None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:26.105871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c6830a06-43d3-4260-bf74-30ea59c9d8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:26.206597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 698/1384] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:26.507584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6012d-14c2-42ea-bf7b-93e446694e8f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:26.509234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c9a6012d-14c2-42ea-bf7b-93e446694e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:26.664947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6012d-14c2-42ea-bf7b-93e446694e8f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:26.743708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:26.792159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206dc7f-958d-4d93-82c4-e1042feb76f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:26.792833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206dc7f-958d-4d93-82c4-e1042feb76f7 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:26.793063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206dc7f-958d-4d93-82c4-e1042feb76f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:26.793341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206dc7f-958d-4d93-82c4-e1042feb76f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:26.795200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f206dc7f-958d-4d93-82c4-e1042feb76f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:26.885824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206dc7f-958d-4d93-82c4-e1042feb76f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:26.887548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206dc7f-958d-4d93-82c4-e1042feb76f7 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:26.907151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 687/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:29.240430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d0424-d5e6-4cb1-87ee-95ea17c3a4cd None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:29.240960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d0424-d5e6-4cb1-87ee-95ea17c3a4cd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:29.240960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d0424-d5e6-4cb1-87ee-95ea17c3a4cd None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:29.242335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 699/1386] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:29.249091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58a09-b29e-400e-a850-040e8e4fac15 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:29.249623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58a09-b29e-400e-a850-040e8e4fac15 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:29.249730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f58a09-b29e-400e-a850-040e8e4fac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:29.340230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f2f58a09-b29e-400e-a850-040e8e4fac15 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:29.342264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f2f58a09-b29e-400e-a850-040e8e4fac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:29.385170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:29.471529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 688/1387] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:57:29.893848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d0424-d5e6-4cb1-87ee-95ea17c3a4cd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:29.895438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f56d0424-d5e6-4cb1-87ee-95ea17c3a4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:30.011342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:30.100573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d0424-d5e6-4cb1-87ee-95ea17c3a4cd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:30.247712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fa193-a778-4f9b-978e-371fe1407c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:30.248683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa193-a778-4f9b-978e-371fe1407c98 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:30.249155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa193-a778-4f9b-978e-371fe1407c98 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:30.249621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa193-a778-4f9b-978e-371fe1407c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:30.252460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e27fa193-a778-4f9b-978e-371fe1407c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:30.307713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:30.361631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fa193-a778-4f9b-978e-371fe1407c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:30.363266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27fa193-a778-4f9b-978e-371fe1407c98 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:30.383272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 700/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:57:32.146457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa58b7-29da-4f21-ae67-372533a09e3a None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:32.147019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa58b7-29da-4f21-ae67-372533a09e3a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:32.147019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa58b7-29da-4f21-ae67-372533a09e3a None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:32.148344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 689/1389] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:32.154367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df273d-a44a-47e7-bd58-ed4e46cdcf97 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:32.154828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df273d-a44a-47e7-bd58-ed4e46cdcf97 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:32.154937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df273d-a44a-47e7-bd58-ed4e46cdcf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:32.220569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-44df273d-a44a-47e7-bd58-ed4e46cdcf97 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:32.221769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-44df273d-a44a-47e7-bd58-ed4e46cdcf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:32.317602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 701/1390] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:32.446768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedbb3d-f1be-47a1-8e33-fffa88deba28 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:32.447016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedbb3d-f1be-47a1-8e33-fffa88deba28 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:32.447292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedbb3d-f1be-47a1-8e33-fffa88deba28 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:32.448653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 690/1391] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:32.454669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:32.454960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:32.455264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:32.470818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:32.521117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:32.522358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:32.617600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ba376-b675-4772-8c05-bb1fe771832e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:32.617889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ba376-b675-4772-8c05-bb1fe771832e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:32.618166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ba376-b675-4772-8c05-bb1fe771832e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:32.619580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 691/1392] 10.222.0.48 () {54 vars in 967 bytes} [Mon Aug 22 10:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 10:57:32.620942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 702/1393] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:32.625506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:32.625796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:32.626075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:32.702625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] MFA Rules not processed for user `990660ce5a9f4213a5b42b3824d0b567`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:32.703747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:32.780275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 692/1394] 10.222.0.48 () {58 vars in 1064 bytes} [Mon Aug 22 10:57:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:32.788067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:32.789537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:32.905104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59aa7a-e827-456a-b4e6-ba0576b6d971 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:32.916791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:32.917749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:32.920103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:32.953291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:33.005800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0c36b-a82e-498b-a516-56aac7711559 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990660ce5a9f4213a5b42b3824d0b567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:33.006901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0c36b-a82e-498b-a516-56aac7711559 service glance] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:33.007612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0c36b-a82e-498b-a516-56aac7711559 service glance] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:33.008053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0c36b-a82e-498b-a516-56aac7711559 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:33.010246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0fe0c36b-a82e-498b-a516-56aac7711559 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:33.050655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81440f6-a0c5-4592-ba92-dcbc0ee4eee1 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:33.130557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0c36b-a82e-498b-a516-56aac7711559 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:33.132002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0c36b-a82e-498b-a516-56aac7711559 service glance] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:33.150627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 703/1395] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:33.168425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187691f4-40ed-4a9c-9cdd-03e88d038728 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:33.169603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187691f4-40ed-4a9c-9cdd-03e88d038728 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:33.169883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187691f4-40ed-4a9c-9cdd-03e88d038728 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:33.170151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187691f4-40ed-4a9c-9cdd-03e88d038728 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:33.171766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-187691f4-40ed-4a9c-9cdd-03e88d038728 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:33.189858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:33.262508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187691f4-40ed-4a9c-9cdd-03e88d038728 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:33.264362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187691f4-40ed-4a9c-9cdd-03e88d038728 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:33.284083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 693/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:57:34.046581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2305a-3cff-42a0-a50d-9d81594c6146 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:34.047060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2305a-3cff-42a0-a50d-9d81594c6146 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:34.047164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2305a-3cff-42a0-a50d-9d81594c6146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:34.092413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:34.115201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-55d2305a-3cff-42a0-a50d-9d81594c6146 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:34.116355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-55d2305a-3cff-42a0-a50d-9d81594c6146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:34.209223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 704/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 10:57:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:57:34.257040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952bb3a-d63a-4d52-b0d5-ab0af26fad04 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:34.257519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952bb3a-d63a-4d52-b0d5-ab0af26fad04 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:34.257637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952bb3a-d63a-4d52-b0d5-ab0af26fad04 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:34.259011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 694/1398] 10.222.0.48 () {54 vars in 943 bytes} [Mon Aug 22 10:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:34.265546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a6cc-c735-4296-a17b-fd365111390f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:34.265833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a6cc-c735-4296-a17b-fd365111390f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:34.266103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149a6cc-c735-4296-a17b-fd365111390f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:34.341047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-0149a6cc-c735-4296-a17b-fd365111390f None None] MFA Rules not processed for user `c5e1b8292ac84020b6ac6db471a6e0ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:34.342262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0149a6cc-c735-4296-a17b-fd365111390f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:34.409784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:34.455396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 705/1399] 10.222.0.48 () {58 vars in 1040 bytes} [Mon Aug 22 10:57:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:57:34.465086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952bb3a-d63a-4d52-b0d5-ab0af26fad04 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:34.467194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a952bb3a-d63a-4d52-b0d5-ab0af26fad04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:34.607778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952bb3a-d63a-4d52-b0d5-ab0af26fad04 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:34.616751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:34.709583 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0996a-2992-494e-98ae-d88470143887 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:34.710116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0996a-2992-494e-98ae-d88470143887 service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:34.710492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0996a-2992-494e-98ae-d88470143887 service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:34.710763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0996a-2992-494e-98ae-d88470143887 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:34.712369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-97e0996a-2992-494e-98ae-d88470143887 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:34.794985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0996a-2992-494e-98ae-d88470143887 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:34.796605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0996a-2992-494e-98ae-d88470143887 service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:34.804621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 695/1400] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 10:57:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 10:57:35.376531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9ac4f-1fe0-4529-b859-918a30d9b6e4 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:35.377061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9ac4f-1fe0-4529-b859-918a30d9b6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:35.377276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9ac4f-1fe0-4529-b859-918a30d9b6e4 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:35.378753 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 706/1401] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:35.385112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980fbca-9419-4c94-962d-20b14e5d1cde None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:35.385590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980fbca-9419-4c94-962d-20b14e5d1cde None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:35.385692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980fbca-9419-4c94-962d-20b14e5d1cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:35.444119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:35.463235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-1980fbca-9419-4c94-962d-20b14e5d1cde None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:35.464480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1980fbca-9419-4c94-962d-20b14e5d1cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:35.575676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 696/1402] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:35.870266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9ac4f-1fe0-4529-b859-918a30d9b6e4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:35.872041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f8a9ac4f-1fe0-4529-b859-918a30d9b6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:36.032222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9ac4f-1fe0-4529-b859-918a30d9b6e4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:36.073771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:36.154038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8ae8b-35ef-42cf-ab81-9c6ee2d65b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:36.154595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8ae8b-35ef-42cf-ab81-9c6ee2d65b8c service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:36.154872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8ae8b-35ef-42cf-ab81-9c6ee2d65b8c service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:36.155140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8ae8b-35ef-42cf-ab81-9c6ee2d65b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:36.156776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-16f8ae8b-35ef-42cf-ab81-9c6ee2d65b8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:36.260835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8ae8b-35ef-42cf-ab81-9c6ee2d65b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:36.262512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8ae8b-35ef-42cf-ab81-9c6ee2d65b8c service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:36.283394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 707/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:38.443618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51efdeb-1199-4e5f-9010-f3adb0330df0 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:38.444489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51efdeb-1199-4e5f-9010-f3adb0330df0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:38.444767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51efdeb-1199-4e5f-9010-f3adb0330df0 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:38.446170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 697/1404] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:38.451646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f993d-39c1-4dd1-ba00-ccb67d0aaccc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:38.452474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f993d-39c1-4dd1-ba00-ccb67d0aaccc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:38.452975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f993d-39c1-4dd1-ba00-ccb67d0aaccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:38.517620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:38.526509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b08f993d-39c1-4dd1-ba00-ccb67d0aaccc None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:38.528156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b08f993d-39c1-4dd1-ba00-ccb67d0aaccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:38.637670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 708/1405] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:57:38.939882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51efdeb-1199-4e5f-9010-f3adb0330df0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:38.941419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a51efdeb-1199-4e5f-9010-f3adb0330df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:39.062879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:39.096443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51efdeb-1199-4e5f-9010-f3adb0330df0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:39.230129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b653d9-ba5c-4276-bf52-f923d948f749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:39.230820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b653d9-ba5c-4276-bf52-f923d948f749 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:39.231120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b653d9-ba5c-4276-bf52-f923d948f749 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:39.231367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b653d9-ba5c-4276-bf52-f923d948f749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:39.232958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c2b653d9-ba5c-4276-bf52-f923d948f749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:39.327049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:39.337865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b653d9-ba5c-4276-bf52-f923d948f749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:39.339505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b653d9-ba5c-4276-bf52-f923d948f749 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:39.370339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 698/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:41.511792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305338-c9c2-4f46-8d07-4bc6d66e39bc None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:41.512483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305338-c9c2-4f46-8d07-4bc6d66e39bc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:41.512669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305338-c9c2-4f46-8d07-4bc6d66e39bc None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:41.514511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 709/1407] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:41.519952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae3a4-9757-4d38-b094-b9f81402d866 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:41.520492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae3a4-9757-4d38-b094-b9f81402d866 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:41.520637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae3a4-9757-4d38-b094-b9f81402d866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:41.593972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-abeae3a4-9757-4d38-b094-b9f81402d866 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:41.595614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-abeae3a4-9757-4d38-b094-b9f81402d866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:41.704807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 699/1408] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:41.991521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84305338-c9c2-4f46-8d07-4bc6d66e39bc None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:57:41.994362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-84305338-c9c2-4f46-8d07-4bc6d66e39bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:42.122613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:42.154418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84305338-c9c2-4f46-8d07-4bc6d66e39bc None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:57:42.283440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31fb09-f4e6-4688-91b2-64eb421e156c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:57:42.284114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31fb09-f4e6-4688-91b2-64eb421e156c service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:42.284417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31fb09-f4e6-4688-91b2-64eb421e156c service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:42.284684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31fb09-f4e6-4688-91b2-64eb421e156c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:42.286307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2b31fb09-f4e6-4688-91b2-64eb421e156c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:42.389937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31fb09-f4e6-4688-91b2-64eb421e156c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:57:42.391662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31fb09-f4e6-4688-91b2-64eb421e156c service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:57:42.395644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:42.413583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 710/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:57:51.702839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=74 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:52.431805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:58.291142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa06a47-a07b-46e4-a249-4dc38b83aff7 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:58.291732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa06a47-a07b-46e4-a249-4dc38b83aff7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:58.291732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa06a47-a07b-46e4-a249-4dc38b83aff7 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:58.293650 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 700/1410] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 22 10:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:58.298946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e015e3-247e-43fc-ad57-7c9b6132d441 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:58.299364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e015e3-247e-43fc-ad57-7c9b6132d441 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:58.299479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e015e3-247e-43fc-ad57-7c9b6132d441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:58.374610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-33e015e3-247e-43fc-ad57-7c9b6132d441 None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:58.375816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-33e015e3-247e-43fc-ad57-7c9b6132d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:58.480364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 711/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 22 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:57:58.496786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa138ac-d22d-4632-b107-ff44ab3dd5d7 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:58.497074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa138ac-d22d-4632-b107-ff44ab3dd5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:58.497352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa138ac-d22d-4632-b107-ff44ab3dd5d7 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:58.498709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 701/1412] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 22 10:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:57:58.503387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c166-d1a9-4c44-9ac0-2ef7feb2027d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:57:58.503677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c166-d1a9-4c44-9ac0-2ef7feb2027d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:57:58.503957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63c166-d1a9-4c44-9ac0-2ef7feb2027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:57:58.574961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:57:58.577926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2c63c166-d1a9-4c44-9ac0-2ef7feb2027d None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:57:58.579165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2c63c166-d1a9-4c44-9ac0-2ef7feb2027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:57:58.692438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 712/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 22 10:57:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:01.070407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ad83-3d7a-458e-aba5-70082d74c996 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:01.070971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ad83-3d7a-458e-aba5-70082d74c996 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:01.070971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ad83-3d7a-458e-aba5-70082d74c996 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:01.072322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:01.078066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e2a3b-50d9-4504-af55-1d0d96e9ea86 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:01.078544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e2a3b-50d9-4504-af55-1d0d96e9ea86 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:01.078717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e2a3b-50d9-4504-af55-1d0d96e9ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:01.144391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ab9e2a3b-50d9-4504-af55-1d0d96e9ea86 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:01.145516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ab9e2a3b-50d9-4504-af55-1d0d96e9ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:01.199824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:01.241881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:01.534213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ad83-3d7a-458e-aba5-70082d74c996 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:01.535757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b854ad83-3d7a-458e-aba5-70082d74c996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:01.679391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ad83-3d7a-458e-aba5-70082d74c996 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:01.751853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:01.793164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0a7f2-6210-472a-a981-94b5c575216a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:01.793687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0a7f2-6210-472a-a981-94b5c575216a service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:01.793960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0a7f2-6210-472a-a981-94b5c575216a service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:01.794223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0a7f2-6210-472a-a981-94b5c575216a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:01.795815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-90c0a7f2-6210-472a-a981-94b5c575216a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:01.905379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0a7f2-6210-472a-a981-94b5c575216a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:01.907035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0a7f2-6210-472a-a981-94b5c575216a service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:01.924732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 703/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:02.836096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff349c3-0777-4e58-b11a-4f5173303c4c None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:02.836672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff349c3-0777-4e58-b11a-4f5173303c4c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:02.836672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff349c3-0777-4e58-b11a-4f5173303c4c None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:02.837973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:02.843643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c943d70-bf97-4ca7-81cf-4f8076913610 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:02.844091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c943d70-bf97-4ca7-81cf-4f8076913610 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:02.844215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c943d70-bf97-4ca7-81cf-4f8076913610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:02.919607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6c943d70-bf97-4ca7-81cf-4f8076913610 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:02.920793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6c943d70-bf97-4ca7-81cf-4f8076913610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:02.947276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:03.016102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:03.166613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff349c3-0777-4e58-b11a-4f5173303c4c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:03.168087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cff349c3-0777-4e58-b11a-4f5173303c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:03.311702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff349c3-0777-4e58-b11a-4f5173303c4c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:03.351948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:03.427491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a94645-9c7c-442d-8f1e-1fcd53cc8609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:03.428003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94645-9c7c-442d-8f1e-1fcd53cc8609 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:03.428283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94645-9c7c-442d-8f1e-1fcd53cc8609 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:03.428554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94645-9c7c-442d-8f1e-1fcd53cc8609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:03.430143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a6a94645-9c7c-442d-8f1e-1fcd53cc8609 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:03.517452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a94645-9c7c-442d-8f1e-1fcd53cc8609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:03.519158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a94645-9c7c-442d-8f1e-1fcd53cc8609 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:03.537581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 715/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:04.419656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0dd1-709f-4374-94ba-eb7a8d1e7d84 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:04.420146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0dd1-709f-4374-94ba-eb7a8d1e7d84 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:04.420247 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0dd1-709f-4374-94ba-eb7a8d1e7d84 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:04.421558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:04.426677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a298953-7a10-46e5-8be3-e407af21181b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:04.427066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a298953-7a10-46e5-8be3-e407af21181b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:04.427263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a298953-7a10-46e5-8be3-e407af21181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:04.482366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:04.503501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-1a298953-7a10-46e5-8be3-e407af21181b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:04.504733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1a298953-7a10-46e5-8be3-e407af21181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:04.600702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:04.754491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db0dd1-709f-4374-94ba-eb7a8d1e7d84 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:04.756021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-73db0dd1-709f-4374-94ba-eb7a8d1e7d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:04.905684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db0dd1-709f-4374-94ba-eb7a8d1e7d84 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:04.917716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:05.017574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f6e48-0574-4ace-89bb-ace80eb1625d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:05.018097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f6e48-0574-4ace-89bb-ace80eb1625d service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:05.018405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f6e48-0574-4ace-89bb-ace80eb1625d service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:05.018672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f6e48-0574-4ace-89bb-ace80eb1625d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:05.020251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-778f6e48-0574-4ace-89bb-ace80eb1625d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:05.107186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f6e48-0574-4ace-89bb-ace80eb1625d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:05.108794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f6e48-0574-4ace-89bb-ace80eb1625d service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:05.127100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 706/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:05.989592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c3192-c8c0-4280-9431-33f45e83065d None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:05.990146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c3192-c8c0-4280-9431-33f45e83065d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:05.990146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c3192-c8c0-4280-9431-33f45e83065d None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:05.991493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:05.997606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ed06e-c5cb-4f01-9ebd-920f0a5b230f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:05.998037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ed06e-c5cb-4f01-9ebd-920f0a5b230f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:05.998156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ed06e-c5cb-4f01-9ebd-920f0a5b230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:06.018798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:06.072782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-db1ed06e-c5cb-4f01-9ebd-920f0a5b230f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:06.074604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-db1ed06e-c5cb-4f01-9ebd-920f0a5b230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:06.170315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:06.320864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c3192-c8c0-4280-9431-33f45e83065d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:06.322418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a13c3192-c8c0-4280-9431-33f45e83065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:06.431375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:06.472788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c3192-c8c0-4280-9431-33f45e83065d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:06.583077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24221f44-14e4-481d-86e9-934e2cb4ea53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:06.583674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24221f44-14e4-481d-86e9-934e2cb4ea53 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:06.583948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24221f44-14e4-481d-86e9-934e2cb4ea53 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:06.584213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24221f44-14e4-481d-86e9-934e2cb4ea53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:06.585792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-24221f44-14e4-481d-86e9-934e2cb4ea53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:06.659057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:06.672562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24221f44-14e4-481d-86e9-934e2cb4ea53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:06.674150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24221f44-14e4-481d-86e9-934e2cb4ea53 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:06.693090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 718/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:58:07.562605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ae7ef-cf12-40ea-add5-b6c1a99c1d4a None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:07.563143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ae7ef-cf12-40ea-add5-b6c1a99c1d4a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:07.563143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ae7ef-cf12-40ea-add5-b6c1a99c1d4a None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:07.564515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:07.569475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a33159-dbe9-4cd6-b76f-1e468cb6d79f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:07.569872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a33159-dbe9-4cd6-b76f-1e468cb6d79f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:07.570021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a33159-dbe9-4cd6-b76f-1e468cb6d79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:07.644863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-95a33159-dbe9-4cd6-b76f-1e468cb6d79f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:07.646063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-95a33159-dbe9-4cd6-b76f-1e468cb6d79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:07.741847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:07.895695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ae7ef-cf12-40ea-add5-b6c1a99c1d4a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:07.899204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cc7ae7ef-cf12-40ea-add5-b6c1a99c1d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:07.981564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:08.042915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ae7ef-cf12-40ea-add5-b6c1a99c1d4a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:08.155049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05e10d-84bf-40c0-bfab-5245253043b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:08.155582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05e10d-84bf-40c0-bfab-5245253043b7 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:08.155860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05e10d-84bf-40c0-bfab-5245253043b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:08.156129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05e10d-84bf-40c0-bfab-5245253043b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:08.157712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5c05e10d-84bf-40c0-bfab-5245253043b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:08.203872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:08.244794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05e10d-84bf-40c0-bfab-5245253043b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:08.246558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05e10d-84bf-40c0-bfab-5245253043b7 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:08.264287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 709/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:58:09.121372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb0841-fcc7-4e02-8d14-103eba8b4915 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:09.121898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb0841-fcc7-4e02-8d14-103eba8b4915 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:09.121898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb0841-fcc7-4e02-8d14-103eba8b4915 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:09.123238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:09.128208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5aadff-8547-4a77-b33c-12a4d28f2219 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:09.128602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5aadff-8547-4a77-b33c-12a4d28f2219 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:09.128763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5aadff-8547-4a77-b33c-12a4d28f2219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:09.205700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5a5aadff-8547-4a77-b33c-12a4d28f2219 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:09.206946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5a5aadff-8547-4a77-b33c-12a4d28f2219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:09.302803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:09.458925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb0841-fcc7-4e02-8d14-103eba8b4915 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:09.460394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f3bb0841-fcc7-4e02-8d14-103eba8b4915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:09.508694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:09.602062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb0841-fcc7-4e02-8d14-103eba8b4915 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:09.712734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b7668-6136-4bd5-a801-eaec6c31f8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:09.713846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b7668-6136-4bd5-a801-eaec6c31f8ff service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:09.714122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b7668-6136-4bd5-a801-eaec6c31f8ff service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:09.714674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b7668-6136-4bd5-a801-eaec6c31f8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:09.716267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d29b7668-6136-4bd5-a801-eaec6c31f8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:09.723168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:09.804618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b7668-6136-4bd5-a801-eaec6c31f8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:09.806867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b7668-6136-4bd5-a801-eaec6c31f8ff service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:09.824798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 721/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:10.691145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363a27-0bad-4672-ba88-8621044f788a None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:10.691615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363a27-0bad-4672-ba88-8621044f788a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:10.691816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d363a27-0bad-4672-ba88-8621044f788a None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:10.693080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:10.699067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458be99d-c83e-4a0f-8570-ccc810d352ed None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:10.699507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458be99d-c83e-4a0f-8570-ccc810d352ed None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:10.699622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458be99d-c83e-4a0f-8570-ccc810d352ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:10.779277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-458be99d-c83e-4a0f-8570-ccc810d352ed None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:10.780500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-458be99d-c83e-4a0f-8570-ccc810d352ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:10.848602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:10.881549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:11.036518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363a27-0bad-4672-ba88-8621044f788a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:11.038030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6d363a27-0bad-4672-ba88-8621044f788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:11.055612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:11.187821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d363a27-0bad-4672-ba88-8621044f788a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:11.285895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:11.311682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885a2ed-fcaa-4728-8b89-0ddc9308abf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:11.312263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885a2ed-fcaa-4728-8b89-0ddc9308abf8 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:11.312568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885a2ed-fcaa-4728-8b89-0ddc9308abf8 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:11.313038 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885a2ed-fcaa-4728-8b89-0ddc9308abf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:11.315117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6885a2ed-fcaa-4728-8b89-0ddc9308abf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:11.402450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885a2ed-fcaa-4728-8b89-0ddc9308abf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:11.404095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885a2ed-fcaa-4728-8b89-0ddc9308abf8 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:11.422426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 712/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:12.309406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1261e-c2d7-4596-a571-5497e758f301 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:12.309979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1261e-c2d7-4596-a571-5497e758f301 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:12.309979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1261e-c2d7-4596-a571-5497e758f301 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:12.311384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:12.316355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c14b2-2d7b-44b8-abad-66e6fafdc24c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:12.316776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c14b2-2d7b-44b8-abad-66e6fafdc24c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:12.316907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c14b2-2d7b-44b8-abad-66e6fafdc24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:12.392365 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d31c14b2-2d7b-44b8-abad-66e6fafdc24c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:12.393645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d31c14b2-2d7b-44b8-abad-66e6fafdc24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:12.436155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:12.489759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:58:12.640423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1261e-c2d7-4596-a571-5497e758f301 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:12.641914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-61c1261e-c2d7-4596-a571-5497e758f301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:12.784773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1261e-c2d7-4596-a571-5497e758f301 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:12.838682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:12.899049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04992cb3-78a9-49ec-9a37-a5a674ca1e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:12.899592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04992cb3-78a9-49ec-9a37-a5a674ca1e44 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:12.899872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04992cb3-78a9-49ec-9a37-a5a674ca1e44 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:12.900144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04992cb3-78a9-49ec-9a37-a5a674ca1e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:12.902034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-04992cb3-78a9-49ec-9a37-a5a674ca1e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:12.989463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04992cb3-78a9-49ec-9a37-a5a674ca1e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:12.991091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04992cb3-78a9-49ec-9a37-a5a674ca1e44 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:13.009226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 724/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:13.891639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7dbbe-cc8a-4c05-898f-72c29359b323 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:13.892128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7dbbe-cc8a-4c05-898f-72c29359b323 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:13.892255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7dbbe-cc8a-4c05-898f-72c29359b323 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:13.893544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 10:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:13.898521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b35e2-bf7c-4bb5-b0b5-903e620b08e9 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:13.898933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b35e2-bf7c-4bb5-b0b5-903e620b08e9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:13.899075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b35e2-bf7c-4bb5-b0b5-903e620b08e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:13.975948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-aa7b35e2-bf7c-4bb5-b0b5-903e620b08e9 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:13.977159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-aa7b35e2-bf7c-4bb5-b0b5-903e620b08e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:13.979389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:14.072386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 10:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:14.222011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7dbbe-cc8a-4c05-898f-72c29359b323 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:14.223563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-79d7dbbe-cc8a-4c05-898f-72c29359b323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:14.369848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7dbbe-cc8a-4c05-898f-72c29359b323 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:14.396205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:14.482813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e830306-0396-44d7-9e73-1dcfb8cd0eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:14.483419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e830306-0396-44d7-9e73-1dcfb8cd0eeb service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:14.483692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e830306-0396-44d7-9e73-1dcfb8cd0eeb service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:14.483959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e830306-0396-44d7-9e73-1dcfb8cd0eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:14.485518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6e830306-0396-44d7-9e73-1dcfb8cd0eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:14.573151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e830306-0396-44d7-9e73-1dcfb8cd0eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:14.574776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e830306-0396-44d7-9e73-1dcfb8cd0eeb service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:14.593435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 715/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:16.408259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b216c0-4105-4426-86ff-77276efc9188 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:16.408803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b216c0-4105-4426-86ff-77276efc9188 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:16.408913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b216c0-4105-4426-86ff-77276efc9188 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:16.410174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:16.415425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ce5d5-0681-4f20-b634-d6f8029670fa None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:16.415820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ce5d5-0681-4f20-b634-d6f8029670fa None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:16.416002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ce5d5-0681-4f20-b634-d6f8029670fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:16.456442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:16.481606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ee5ce5d5-0681-4f20-b634-d6f8029670fa None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:16.482875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ee5ce5d5-0681-4f20-b634-d6f8029670fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:16.579123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:16.867336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b216c0-4105-4426-86ff-77276efc9188 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:16.868944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-13b216c0-4105-4426-86ff-77276efc9188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:16.972233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b216c0-4105-4426-86ff-77276efc9188 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:17.003505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:17.044125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6a292-5f16-4775-a8b0-3087914eec23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:17.044727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a292-5f16-4775-a8b0-3087914eec23 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:17.045007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a292-5f16-4775-a8b0-3087914eec23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:17.045281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a292-5f16-4775-a8b0-3087914eec23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:17.046905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9ea6a292-5f16-4775-a8b0-3087914eec23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:17.152880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6a292-5f16-4775-a8b0-3087914eec23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:17.154261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6a292-5f16-4775-a8b0-3087914eec23 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:17.172576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 727/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:19.317896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fe1bd-ec55-4467-b962-4f472b650830 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:19.318509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fe1bd-ec55-4467-b962-4f472b650830 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:19.318618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fe1bd-ec55-4467-b962-4f472b650830 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:19.319955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:19.325353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443dab-d403-42aa-b1dd-869b940215de None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:19.325760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443dab-d403-42aa-b1dd-869b940215de None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:19.325901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58443dab-d403-42aa-b1dd-869b940215de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:19.392795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-58443dab-d403-42aa-b1dd-869b940215de None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:19.394049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-58443dab-d403-42aa-b1dd-869b940215de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:19.407236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:19.499432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:19.790535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fe1bd-ec55-4467-b962-4f472b650830 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:19.792826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2b7fe1bd-ec55-4467-b962-4f472b650830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:19.896199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fe1bd-ec55-4467-b962-4f472b650830 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:19.910482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:19.970688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e4fad-331a-4466-888b-ac1f437c65b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:19.971291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e4fad-331a-4466-888b-ac1f437c65b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:19.971569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e4fad-331a-4466-888b-ac1f437c65b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:19.971839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e4fad-331a-4466-888b-ac1f437c65b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:19.973432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-809e4fad-331a-4466-888b-ac1f437c65b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:20.080020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e4fad-331a-4466-888b-ac1f437c65b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:20.081351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e4fad-331a-4466-888b-ac1f437c65b7 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:20.098675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 718/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:22.401708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d3fc-d42b-4996-a9cc-2fbc6444e7fb None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:22.402290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d3fc-d42b-4996-a9cc-2fbc6444e7fb None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:22.402290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d3fc-d42b-4996-a9cc-2fbc6444e7fb None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:22.403654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:22.408932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe18de-49f7-4788-8aba-89677c442ef0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:22.409374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe18de-49f7-4788-8aba-89677c442ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:22.409478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe18de-49f7-4788-8aba-89677c442ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:22.458104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:22.475117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ddfe18de-49f7-4788-8aba-89677c442ef0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:22.476320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ddfe18de-49f7-4788-8aba-89677c442ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:22.570077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:22.858933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00d3fc-d42b-4996-a9cc-2fbc6444e7fb None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:22.860569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ba00d3fc-d42b-4996-a9cc-2fbc6444e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:22.963594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00d3fc-d42b-4996-a9cc-2fbc6444e7fb None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:23.003246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:23.036630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c13f8-659b-461d-9dce-7a99a4224acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:23.037230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c13f8-659b-461d-9dce-7a99a4224acf service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:23.037510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c13f8-659b-461d-9dce-7a99a4224acf service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:23.037783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c13f8-659b-461d-9dce-7a99a4224acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:23.039414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-740c13f8-659b-461d-9dce-7a99a4224acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:23.146032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c13f8-659b-461d-9dce-7a99a4224acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:23.147412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c13f8-659b-461d-9dce-7a99a4224acf service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:23.166528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 730/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:25.177373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05124-96e9-4acb-8ddd-a7cb17331f49 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:25.177909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05124-96e9-4acb-8ddd-a7cb17331f49 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:25.177909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05124-96e9-4acb-8ddd-a7cb17331f49 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:25.179263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:25.185228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6ebc-503a-422d-9028-4be48e551e90 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:25.185632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6ebc-503a-422d-9028-4be48e551e90 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:25.185780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6ebc-503a-422d-9028-4be48e551e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:25.250324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-562d6ebc-503a-422d-9028-4be48e551e90 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:25.251532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-562d6ebc-503a-422d-9028-4be48e551e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:25.271702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:25.348419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:58:25.638767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05124-96e9-4acb-8ddd-a7cb17331f49 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:25.640297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d1d05124-96e9-4acb-8ddd-a7cb17331f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:25.741602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d05124-96e9-4acb-8ddd-a7cb17331f49 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:25.761239 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:25.815808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33d39f-4a72-4415-b107-d6f8f94a8d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:25.816421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33d39f-4a72-4415-b107-d6f8f94a8d82 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:25.816698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33d39f-4a72-4415-b107-d6f8f94a8d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:25.816964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33d39f-4a72-4415-b107-d6f8f94a8d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:25.818571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-af33d39f-4a72-4415-b107-d6f8f94a8d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:25.929278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33d39f-4a72-4415-b107-d6f8f94a8d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:25.930682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33d39f-4a72-4415-b107-d6f8f94a8d82 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:25.949061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 721/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:27.965927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540fbcb-bb1c-448f-a5ae-1225d1d6e0cd None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:27.966507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540fbcb-bb1c-448f-a5ae-1225d1d6e0cd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:27.966507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540fbcb-bb1c-448f-a5ae-1225d1d6e0cd None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:27.967859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:27.973389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9ab9-3a5e-490c-90a5-f06657466a9b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:27.973787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9ab9-3a5e-490c-90a5-f06657466a9b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:27.973961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9ab9-3a5e-490c-90a5-f06657466a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:28.038828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:28.041562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c3ef9ab9-3a5e-490c-90a5-f06657466a9b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:28.042809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c3ef9ab9-3a5e-490c-90a5-f06657466a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:28.138824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:28.426355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540fbcb-bb1c-448f-a5ae-1225d1d6e0cd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:28.427962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8540fbcb-bb1c-448f-a5ae-1225d1d6e0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:28.531233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540fbcb-bb1c-448f-a5ae-1225d1d6e0cd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:28.578602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:28.603457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc9c7-4e3a-4125-a434-7f48328d6c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:28.603926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc9c7-4e3a-4125-a434-7f48328d6c05 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:28.604200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc9c7-4e3a-4125-a434-7f48328d6c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:28.604467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc9c7-4e3a-4125-a434-7f48328d6c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:28.606475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-337bc9c7-4e3a-4125-a434-7f48328d6c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:28.711633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc9c7-4e3a-4125-a434-7f48328d6c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:28.713660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc9c7-4e3a-4125-a434-7f48328d6c05 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:28.732729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 733/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:30.810300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f36f8-dd92-42ab-b7a4-f3114350ae9d None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:30.810867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f36f8-dd92-42ab-b7a4-f3114350ae9d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:30.810867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f36f8-dd92-42ab-b7a4-f3114350ae9d None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:30.812194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:30.817559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47abc3-9307-4226-be61-7b0cb4957586 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:30.817988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47abc3-9307-4226-be61-7b0cb4957586 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:30.818120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47abc3-9307-4226-be61-7b0cb4957586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:30.884868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-0d47abc3-9307-4226-be61-7b0cb4957586 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:30.886222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0d47abc3-9307-4226-be61-7b0cb4957586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:30.915345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:30.988164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:31.281951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f36f8-dd92-42ab-b7a4-f3114350ae9d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:31.283544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e58f36f8-dd92-42ab-b7a4-f3114350ae9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:31.387121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f36f8-dd92-42ab-b7a4-f3114350ae9d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:31.415753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:31.461517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea03c1-2a48-4327-bf5b-23956cf4c7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:31.462921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea03c1-2a48-4327-bf5b-23956cf4c7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:31.463201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea03c1-2a48-4327-bf5b-23956cf4c7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:31.463470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea03c1-2a48-4327-bf5b-23956cf4c7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:31.465033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1aea03c1-2a48-4327-bf5b-23956cf4c7c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:31.579202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea03c1-2a48-4327-bf5b-23956cf4c7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:31.580566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea03c1-2a48-4327-bf5b-23956cf4c7c2 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:31.599537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 724/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:33.554875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cb03a-455b-44e6-848d-61fff87a3f80 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:33.555466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cb03a-455b-44e6-848d-61fff87a3f80 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:33.555466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cb03a-455b-44e6-848d-61fff87a3f80 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:33.557547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:33.563295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2c9a-5353-42f6-b41a-5c0d408095c7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:33.563700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2c9a-5353-42f6-b41a-5c0d408095c7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:33.563874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a2c9a-5353-42f6-b41a-5c0d408095c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:33.629503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-365a2c9a-5353-42f6-b41a-5c0d408095c7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:33.630726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-365a2c9a-5353-42f6-b41a-5c0d408095c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:33.639464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:33.728802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:34.013907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cb03a-455b-44e6-848d-61fff87a3f80 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:34.015470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c04cb03a-455b-44e6-848d-61fff87a3f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:34.233962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cb03a-455b-44e6-848d-61fff87a3f80 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:34.288403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:34.306275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416feee-bf55-4e5c-ab08-ed60c1762907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:34.311771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416feee-bf55-4e5c-ab08-ed60c1762907 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:34.311771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416feee-bf55-4e5c-ab08-ed60c1762907 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:34.311771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416feee-bf55-4e5c-ab08-ed60c1762907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:34.311771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b416feee-bf55-4e5c-ab08-ed60c1762907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:34.413572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416feee-bf55-4e5c-ab08-ed60c1762907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:34.415631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416feee-bf55-4e5c-ab08-ed60c1762907 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:34.434691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 736/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:36.403834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7da3-1c39-4152-baf0-b6f50eac1c8d None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:36.404426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7da3-1c39-4152-baf0-b6f50eac1c8d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:36.404704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d7da3-1c39-4152-baf0-b6f50eac1c8d None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:36.406104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:36.411613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39135-fe16-4aea-8071-111f7f362e07 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:36.412038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39135-fe16-4aea-8071-111f7f362e07 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:36.412166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39135-fe16-4aea-8071-111f7f362e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:36.478905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-fcb39135-fe16-4aea-8071-111f7f362e07 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:36.480119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-fcb39135-fe16-4aea-8071-111f7f362e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:36.517556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:36.575274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:36.864174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7da3-1c39-4152-baf0-b6f50eac1c8d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:36.865734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-958d7da3-1c39-4152-baf0-b6f50eac1c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:36.968822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d7da3-1c39-4152-baf0-b6f50eac1c8d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:37.005659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:37.040306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687bd88-927e-4d9a-b49b-b02292f38b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:37.040906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687bd88-927e-4d9a-b49b-b02292f38b8c service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:37.041182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687bd88-927e-4d9a-b49b-b02292f38b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:37.041452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687bd88-927e-4d9a-b49b-b02292f38b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:37.043449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8687bd88-927e-4d9a-b49b-b02292f38b8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:37.152490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687bd88-927e-4d9a-b49b-b02292f38b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:37.152490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687bd88-927e-4d9a-b49b-b02292f38b8c service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:37.169978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 727/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:39.098305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6123a-30fc-4189-8d38-e392ebf8bdab None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:39.098859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6123a-30fc-4189-8d38-e392ebf8bdab None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:39.098859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6123a-30fc-4189-8d38-e392ebf8bdab None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:39.100126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:39.106303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c5d55-ead6-4e67-9f0b-35786b07f3b0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:39.106744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c5d55-ead6-4e67-9f0b-35786b07f3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:39.106853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c5d55-ead6-4e67-9f0b-35786b07f3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:39.173168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ae0c5d55-ead6-4e67-9f0b-35786b07f3b0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:39.174396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ae0c5d55-ead6-4e67-9f0b-35786b07f3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:39.192059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:39.271714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:58:39.557390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6123a-30fc-4189-8d38-e392ebf8bdab None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:39.558976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-25f6123a-30fc-4189-8d38-e392ebf8bdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:39.660874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6123a-30fc-4189-8d38-e392ebf8bdab None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:39.728222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:39.734464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3506-4c39-48c8-a4ea-325152dde89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:39.735063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3506-4c39-48c8-a4ea-325152dde89d service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:39.735336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3506-4c39-48c8-a4ea-325152dde89d service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:39.735602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3506-4c39-48c8-a4ea-325152dde89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:39.737171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3e9c3506-4c39-48c8-a4ea-325152dde89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:39.844048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c3506-4c39-48c8-a4ea-325152dde89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:39.845493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c3506-4c39-48c8-a4ea-325152dde89d service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:39.864193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 739/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:41.796717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a3495-fec8-4e02-846e-393c9404dc2b None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:41.797298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a3495-fec8-4e02-846e-393c9404dc2b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:41.797298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a3495-fec8-4e02-846e-393c9404dc2b None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:41.798669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 10:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:58:41.804114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc20352-cedb-443b-aee9-100cfe640c01 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:41.804520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc20352-cedb-443b-aee9-100cfe640c01 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:41.804664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc20352-cedb-443b-aee9-100cfe640c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:41.871383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-acc20352-cedb-443b-aee9-100cfe640c01 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:58:41.872587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-acc20352-cedb-443b-aee9-100cfe640c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:41.927396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:41.975273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 10:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:58:42.261710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a3495-fec8-4e02-846e-393c9404dc2b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:58:42.263286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-203a3495-fec8-4e02-846e-393c9404dc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:42.365360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a3495-fec8-4e02-846e-393c9404dc2b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:58:42.413451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:42.440263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627434ad-ef01-4dd8-8a41-9b9a6ea6de59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:58:42.440875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627434ad-ef01-4dd8-8a41-9b9a6ea6de59 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:58:42.441154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627434ad-ef01-4dd8-8a41-9b9a6ea6de59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:58:42.441427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627434ad-ef01-4dd8-8a41-9b9a6ea6de59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:58:42.443095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-627434ad-ef01-4dd8-8a41-9b9a6ea6de59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:58:42.553684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627434ad-ef01-4dd8-8a41-9b9a6ea6de59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:58:42.555668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627434ad-ef01-4dd8-8a41-9b9a6ea6de59 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:58:42.577802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 730/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:58:51.976653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=22 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:58:52.575454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=66 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:02.990003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbb9a0-1b09-4876-8577-88d540c3843f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:02.990580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbb9a0-1b09-4876-8577-88d540c3843f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:02.990580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbb9a0-1b09-4876-8577-88d540c3843f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:02.992475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 741/1471] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:02.998005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb88b8-2509-4cef-b03b-a66860f3b1d2 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:02.998483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb88b8-2509-4cef-b03b-a66860f3b1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:02.998595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb88b8-2509-4cef-b03b-a66860f3b1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:03.064814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6dfb88b8-2509-4cef-b03b-a66860f3b1d2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:03.066021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6dfb88b8-2509-4cef-b03b-a66860f3b1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:03.160248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 731/1472] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:59:03.456792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5745b7-8fd6-4f7f-8e4b-f6df491b2965 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:03.457084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5745b7-8fd6-4f7f-8e4b-f6df491b2965 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:03.457356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5745b7-8fd6-4f7f-8e4b-f6df491b2965 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:03.458724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 742/1473] 10.222.0.48 () {54 vars in 967 bytes} [Mon Aug 22 10:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:03.464161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae89e-3370-46f6-bf22-fc1336229017 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:03.467347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae89e-3370-46f6-bf22-fc1336229017 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:03.467347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ae89e-3370-46f6-bf22-fc1336229017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:03.542820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9f8ae89e-3370-46f6-bf22-fc1336229017 None None] MFA Rules not processed for user `990660ce5a9f4213a5b42b3824d0b567`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:03.544042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9f8ae89e-3370-46f6-bf22-fc1336229017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:03.564478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:03.621737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 732/1474] 10.222.0.48 () {58 vars in 1064 bytes} [Mon Aug 22 10:59:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:59:03.629743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5745b7-8fd6-4f7f-8e4b-f6df491b2965 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:59:03.631317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-da5745b7-8fd6-4f7f-8e4b-f6df491b2965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:03.735512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5745b7-8fd6-4f7f-8e4b-f6df491b2965 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:59:03.808197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fc4ed-2d70-45b7-8772-d045b02165c5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990660ce5a9f4213a5b42b3824d0b567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:59:03.808810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fc4ed-2d70-45b7-8772-d045b02165c5 service glance] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:03.809086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fc4ed-2d70-45b7-8772-d045b02165c5 service glance] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:03.809354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fc4ed-2d70-45b7-8772-d045b02165c5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:03.811011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-858fc4ed-2d70-45b7-8772-d045b02165c5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:03.873408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:03.928286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fc4ed-2d70-45b7-8772-d045b02165c5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:59:03.929704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fc4ed-2d70-45b7-8772-d045b02165c5 service glance] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:59:03.949334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 743/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:59:05.946937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc7cb-4e16-4143-a60f-8606d4fd0330 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:05.947478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc7cb-4e16-4143-a60f-8606d4fd0330 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:05.947578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bc7cb-4e16-4143-a60f-8606d4fd0330 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:05.948874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 733/1476] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:05.954125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643761a-bd3f-4dd6-b519-a85b2840bf06 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:05.954624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643761a-bd3f-4dd6-b519-a85b2840bf06 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:05.954726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643761a-bd3f-4dd6-b519-a85b2840bf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:06.019321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d643761a-bd3f-4dd6-b519-a85b2840bf06 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:06.020563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d643761a-bd3f-4dd6-b519-a85b2840bf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:06.114637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 744/1477] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:59:06.393513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bc7cb-4e16-4143-a60f-8606d4fd0330 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:59:06.395015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8e4bc7cb-4e16-4143-a60f-8606d4fd0330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:06.520683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bc7cb-4e16-4143-a60f-8606d4fd0330 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:59:06.569654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:06.613735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233b2e8-b8b3-47ab-aa98-2f90d60fd75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:59:06.614374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233b2e8-b8b3-47ab-aa98-2f90d60fd75a admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:06.614657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233b2e8-b8b3-47ab-aa98-2f90d60fd75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:06.614927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233b2e8-b8b3-47ab-aa98-2f90d60fd75a admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:06.615566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233b2e8-b8b3-47ab-aa98-2f90d60fd75a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:59:06.617348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-d233b2e8-b8b3-47ab-aa98-2f90d60fd75a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 10:59:06.617348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 10:59:06.617348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 10:59:06.619366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233b2e8-b8b3-47ab-aa98-2f90d60fd75a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:59:06.637876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 734/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 22 10:59:06 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 10:59:08.561239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f7835-0863-46a3-b8eb-400787133ef6 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:08.561768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f7835-0863-46a3-b8eb-400787133ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:08.561768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f7835-0863-46a3-b8eb-400787133ef6 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:08.563149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 745/1479] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:08.568381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f364f4-8d1a-42df-a8d8-6d0ac73c44a8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:08.568810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f364f4-8d1a-42df-a8d8-6d0ac73c44a8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:08.568908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f364f4-8d1a-42df-a8d8-6d0ac73c44a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:08.635013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-43f364f4-8d1a-42df-a8d8-6d0ac73c44a8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:08.636197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-43f364f4-8d1a-42df-a8d8-6d0ac73c44a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:08.849721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 735/1480] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 10:59:09.143406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f7835-0863-46a3-b8eb-400787133ef6 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:59:09.144929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a41f7835-0863-46a3-b8eb-400787133ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:09.155364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:09.291282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f7835-0863-46a3-b8eb-400787133ef6 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:59:09.371845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:09.404849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bd17c-e72e-4bf3-af73-81385495d53c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:59:09.405388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd17c-e72e-4bf3-af73-81385495d53c service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:09.405663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd17c-e72e-4bf3-af73-81385495d53c service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:09.405933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bd17c-e72e-4bf3-af73-81385495d53c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:09.407550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0f4bd17c-e72e-4bf3-af73-81385495d53c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:09.495895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bd17c-e72e-4bf3-af73-81385495d53c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:59:09.497470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bd17c-e72e-4bf3-af73-81385495d53c service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:59:09.515465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 746/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:59:11.592043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2c19-1890-4e62-b1ed-3004ac454208 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:11.592607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2c19-1890-4e62-b1ed-3004ac454208 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:11.592607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f2c19-1890-4e62-b1ed-3004ac454208 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:11.593941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 736/1482] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:11.599184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d892-a2d8-492a-9772-641ed4530532 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:11.599611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d892-a2d8-492a-9772-641ed4530532 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:11.599735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d892-a2d8-492a-9772-641ed4530532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:11.665126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-de19d892-a2d8-492a-9772-641ed4530532 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:11.666348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-de19d892-a2d8-492a-9772-641ed4530532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:11.713689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:11.763063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 747/1483] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:59:12.058908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f2c19-1890-4e62-b1ed-3004ac454208 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:59:12.060452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-299f2c19-1890-4e62-b1ed-3004ac454208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:12.083251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:12.162763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f2c19-1890-4e62-b1ed-3004ac454208 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:59:12.247805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda20592-6b53-4854-b89b-30bfa1bd7cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:59:12.248414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20592-6b53-4854-b89b-30bfa1bd7cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:12.248689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20592-6b53-4854-b89b-30bfa1bd7cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:12.248954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20592-6b53-4854-b89b-30bfa1bd7cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:12.250628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bda20592-6b53-4854-b89b-30bfa1bd7cd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:12.323291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:12.357608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda20592-6b53-4854-b89b-30bfa1bd7cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:59:12.359085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda20592-6b53-4854-b89b-30bfa1bd7cd5 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:59:12.377752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 737/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:59:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 10:59:14.345281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adff4d-6f7b-4330-975e-da36aabd7b5f None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:14.345836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adff4d-6f7b-4330-975e-da36aabd7b5f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:14.345836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93adff4d-6f7b-4330-975e-da36aabd7b5f None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:14.347261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 748/1485] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:14.353273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d056-0a97-4494-8035-439bb6e371ce None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:14.353682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d056-0a97-4494-8035-439bb6e371ce None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:14.353809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7d056-0a97-4494-8035-439bb6e371ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:14.419114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ece7d056-0a97-4494-8035-439bb6e371ce None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:14.420322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ece7d056-0a97-4494-8035-439bb6e371ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:14.516417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 738/1486] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:59:14.805142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adff4d-6f7b-4330-975e-da36aabd7b5f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:59:14.806706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-93adff4d-6f7b-4330-975e-da36aabd7b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:14.911589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93adff4d-6f7b-4330-975e-da36aabd7b5f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:59:14.983771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:14.985561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e65fef-7b56-4aa4-8eec-813d6e35d7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:59:14.986165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e65fef-7b56-4aa4-8eec-813d6e35d7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:14.986469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e65fef-7b56-4aa4-8eec-813d6e35d7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:14.986740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e65fef-7b56-4aa4-8eec-813d6e35d7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:14.988330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-14e65fef-7b56-4aa4-8eec-813d6e35d7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:15.095995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e65fef-7b56-4aa4-8eec-813d6e35d7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:59:15.097336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e65fef-7b56-4aa4-8eec-813d6e35d7d3 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:59:15.115119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 749/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:59:17.115146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2029a8-7010-44f8-9878-2b81db8f0a10 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:17.115671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2029a8-7010-44f8-9878-2b81db8f0a10 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:17.115671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2029a8-7010-44f8-9878-2b81db8f0a10 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:17.117136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 739/1488] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:17.122921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57e78b-70b5-4b93-86fe-bc174cbe165b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:17.123362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57e78b-70b5-4b93-86fe-bc174cbe165b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:17.123475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57e78b-70b5-4b93-86fe-bc174cbe165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:17.188725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-0b57e78b-70b5-4b93-86fe-bc174cbe165b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:17.189922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0b57e78b-70b5-4b93-86fe-bc174cbe165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:17.241238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:17.285381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 750/1489] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:59:17.586042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2029a8-7010-44f8-9878-2b81db8f0a10 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:59:17.587687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ce2029a8-7010-44f8-9878-2b81db8f0a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:17.624269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:17.691033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2029a8-7010-44f8-9878-2b81db8f0a10 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:59:17.765718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b2265-17c2-41a7-b51d-d0f782e56528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:59:17.766376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2265-17c2-41a7-b51d-d0f782e56528 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:17.766654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2265-17c2-41a7-b51d-d0f782e56528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:17.766924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2265-17c2-41a7-b51d-d0f782e56528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:17.768505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-519b2265-17c2-41a7-b51d-d0f782e56528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:17.855497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:17.884659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b2265-17c2-41a7-b51d-d0f782e56528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:59:17.886096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b2265-17c2-41a7-b51d-d0f782e56528 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:59:17.904219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 740/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:59:20.143548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f9743-1d73-4a1e-a2b4-7658ad873d0a None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:20.144161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f9743-1d73-4a1e-a2b4-7658ad873d0a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:20.144161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f9743-1d73-4a1e-a2b4-7658ad873d0a None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:20.145386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 751/1491] 10.222.0.48 () {54 vars in 927 bytes} [Mon Aug 22 10:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 10:59:20.151282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ef871-7f28-442c-9229-f55228238d46 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:20.151685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ef871-7f28-442c-9229-f55228238d46 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:20.151840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ef871-7f28-442c-9229-f55228238d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:20.216838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-464ef871-7f28-442c-9229-f55228238d46 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 10:59:20.218045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-464ef871-7f28-442c-9229-f55228238d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:20.312428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 741/1492] 10.222.0.48 () {58 vars in 1024 bytes} [Mon Aug 22 10:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 10:59:20.599287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f9743-1d73-4a1e-a2b4-7658ad873d0a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 10:59:20.600822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-aa0f9743-1d73-4a1e-a2b4-7658ad873d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:20.701714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f9743-1d73-4a1e-a2b4-7658ad873d0a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 10:59:20.773327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c8a9a-2d0b-46fc-b194-1d9cd5cfaffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 10:59:20.773982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c8a9a-2d0b-46fc-b194-1d9cd5cfaffb service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 10:59:20.774289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c8a9a-2d0b-46fc-b194-1d9cd5cfaffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 10:59:20.774590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c8a9a-2d0b-46fc-b194-1d9cd5cfaffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 10:59:20.776179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-388c8a9a-2d0b-46fc-b194-1d9cd5cfaffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 10:59:20.785113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:20.885498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c8a9a-2d0b-46fc-b194-1d9cd5cfaffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 10:59:20.886969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c8a9a-2d0b-46fc-b194-1d9cd5cfaffb service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 10:59:20.905515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 752/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 10:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 10:59:30.310631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=82 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 10:59:30.903282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=50 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:41.663597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:41.664264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:41.664376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:41.744015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:00:41.745223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:41.842124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 742/1494] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:00:41.850868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7b9a4-4c68-481e-937c-0d19b7c17f1d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:41.851387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7b9a4-4c68-481e-937c-0d19b7c17f1d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:41.851496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7b9a4-4c68-481e-937c-0d19b7c17f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:41.928029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-73a7b9a4-4c68-481e-937c-0d19b7c17f1d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:00:41.929163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-73a7b9a4-4c68-481e-937c-0d19b7c17f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:42.026116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 753/1495] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:00:42.030836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:42.032051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:42.043289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f604367-2ecf-4a7e-afbe-3c0f2d720b9a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:42.103842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:42.138494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:42.139104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:42.139384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:42.139661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:42.140298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:00:42.140598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:42.141606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:42.141606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 11:00:42.141606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 11:00:42.142508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:42.153429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 743/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:00:42.161679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7b9a4-4c68-481e-937c-0d19b7c17f1d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:42.170725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7b9a4-4c68-481e-937c-0d19b7c17f1d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:42.230959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:42.264866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:42.265467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:42.265747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:42.266022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:42.266669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:42.268407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING py.warnings [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:42.268407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: warnings.warn(msg) Aug 22 11:00:42.268407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: Aug 22 11:00:42.270293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:42.388070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 754/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:00:42 2022] POST /identity/v3/projects => generated 375 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:00:42.392382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:42.394568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:42.426353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07c648-28d7-4a8b-9870-3f59417ad85d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:42.517773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:42.518415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:42.519290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:42.519557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:42.520265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:42.521267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:42.521267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 11:00:42.521267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 11:00:42.522485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:42.630952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 744/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:00:42 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:00:42.635169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:42.636684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:42.697181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1dcb5-5eae-4287-a83c-f2cf9fe98bde admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:42.777783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:42.787920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:42.789560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:42.789560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:42.789560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:42.789848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:42.790731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING py.warnings [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:42.790731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: warnings.warn(msg) Aug 22 11:00:42.790731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: Aug 22 11:00:42.791608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:42.804426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 755/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:00:42.808289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:42.809791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:42.821708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28326f0-7193-4018-b53d-b704c8cbf7d0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:42.846017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:42.916350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f56ba-a552-44b6-bb48-752dfa5eeb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:42.916956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f56ba-a552-44b6-bb48-752dfa5eeb05 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:42.917227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f56ba-a552-44b6-bb48-752dfa5eeb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:42.917492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f56ba-a552-44b6-bb48-752dfa5eeb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:42.918119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f56ba-a552-44b6-bb48-752dfa5eeb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:42.919167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-fa2f56ba-a552-44b6-bb48-752dfa5eeb05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:42.919167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 11:00:42.919167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 11:00:42.920028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f56ba-a552-44b6-bb48-752dfa5eeb05 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:42.932654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 745/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:00:42.936838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:42.938431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:42.971461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a3c6a-a9a8-4cfe-832a-162803a83bc6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:43.065579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5478f8a-9bd8-489e-b538-f106e0302fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:43.066428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5478f8a-9bd8-489e-b538-f106e0302fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:43.066715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5478f8a-9bd8-489e-b538-f106e0302fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:43.067020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5478f8a-9bd8-489e-b538-f106e0302fdb admin admin] PATH_INFO: `/v3/projects/f366d4452d0c4c81b33da1b2c71ccef8/users/65408a3e23254ac0991a1ec82a770127/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:43.088618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5478f8a-9bd8-489e-b538-f106e0302fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f366d4452d0c4c81b33da1b2c71ccef8, user_id=65408a3e23254ac0991a1ec82a770127, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:43.089648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING py.warnings [None req-b5478f8a-9bd8-489e-b538-f106e0302fdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:43.089648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: warnings.warn(msg) Aug 22 11:00:43.089648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: Aug 22 11:00:43.091599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5478f8a-9bd8-489e-b538-f106e0302fdb admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:43.181643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 756/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:00:42 2022] PUT /identity/v3/projects/f366d4452d0c4c81b33da1b2c71ccef8/users/65408a3e23254ac0991a1ec82a770127/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:00:43.194154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba11fd-611d-4e53-b4a6-319177aaf591 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:43.194476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba11fd-611d-4e53-b4a6-319177aaf591 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:43.194753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba11fd-611d-4e53-b4a6-319177aaf591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:43.251154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-11ba11fd-611d-4e53-b4a6-319177aaf591 None None] MFA Rules not processed for user `65408a3e23254ac0991a1ec82a770127`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:00:43.252344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-11ba11fd-611d-4e53-b4a6-319177aaf591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:43.326843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 746/1502] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:00:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:00:43.339952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a921fd-6674-4e43-8127-66d6dcceb4f4 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:43.340234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a921fd-6674-4e43-8127-66d6dcceb4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:43.340506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a921fd-6674-4e43-8127-66d6dcceb4f4 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:43.341826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 757/1503] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 22 11:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:00:43.346464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b838d-6ffe-4e24-ba3f-1ef99615e3c3 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:43.347275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b838d-6ffe-4e24-ba3f-1ef99615e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:43.347542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b838d-6ffe-4e24-ba3f-1ef99615e3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:43.378158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:43.422458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e08b838d-6ffe-4e24-ba3f-1ef99615e3c3 None None] MFA Rules not processed for user `7cdb62860b6243bb84c31d1caff4bc77`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:00:43.423653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e08b838d-6ffe-4e24-ba3f-1ef99615e3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:43.498391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 747/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 22 11:00:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:00:43.506192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a921fd-6674-4e43-8127-66d6dcceb4f4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:43.507736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-14a921fd-6674-4e43-8127-66d6dcceb4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:43.557787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:43.610485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a921fd-6674-4e43-8127-66d6dcceb4f4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:43.683779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:43.684383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:43.684663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:43.684935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:43.686554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:43.785794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:43.787257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:43.790819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:43.805312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 758/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:00:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:00:43.850529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea503-884f-4b2e-a712-84f34ab73579 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:43.851000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea503-884f-4b2e-a712-84f34ab73579 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:43.851098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea503-884f-4b2e-a712-84f34ab73579 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:43.852347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 748/1506] 10.222.0.48 () {52 vars in 830 bytes} [Mon Aug 22 11:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:00:43.855993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:43.857208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:43.954501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309e681-2943-4429-8179-85e31f7e92d1 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:44.049449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:44.050707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:44.051123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:44.051395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] PATH_INFO: `/v3/users/65408a3e23254ac0991a1ec82a770127` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:44.063303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=65408a3e23254ac0991a1ec82a770127)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:44.064324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING py.warnings [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:44.064324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: warnings.warn(msg) Aug 22 11:00:44.064324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: Aug 22 11:00:44.065611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:44.075372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:44.249980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65408a3e23254ac0991a1ec82a770127'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:44.250540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65408a3e23254ac0991a1ec82a770127'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:44.250961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65408a3e23254ac0991a1ec82a770127'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:44.265491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65408a3e23254ac0991a1ec82a770127'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:44.325863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65408a3e23254ac0991a1ec82a770127'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:44.354282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 759/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:00:43 2022] DELETE /identity/v3/users/65408a3e23254ac0991a1ec82a770127 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:00:44.358640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:44.358931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:44.359262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:44.416569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:00:44.417879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:44.492922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:44.506540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 749/1508] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:00:44.515300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:44.516513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:44.622133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9828a27-5724-4e1d-9695-2202fa8bea6a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:44.678334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:44.697240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b96a50-8514-4657-b35b-db890ccd4aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:44.697849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b96a50-8514-4657-b35b-db890ccd4aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:44.698131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b96a50-8514-4657-b35b-db890ccd4aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:44.698442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b96a50-8514-4657-b35b-db890ccd4aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:44.700037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c8b96a50-8514-4657-b35b-db890ccd4aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:44.812506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b96a50-8514-4657-b35b-db890ccd4aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:44.813857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b96a50-8514-4657-b35b-db890ccd4aaf service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:44.835160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 760/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:00:45.351315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:00:45.352655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:00:45.364225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2337122-3773-4b31-b749-010a1edf4514 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:00:45.470661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:00:45.471301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:00:45.471623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:00:45.471896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] PATH_INFO: `/v3/projects/f366d4452d0c4c81b33da1b2c71ccef8` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:00:45.481867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f366d4452d0c4c81b33da1b2c71ccef8)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:00:45.482923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:00:45.482923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 11:00:45.482923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 11:00:45.484180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:00:45.598720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f366d4452d0c4c81b33da1b2c71ccef8', 'request_id': 'req-f040fcae-4c4e-4407-b498-b1615a803d69'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:45.604727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f366d4452d0c4c81b33da1b2c71ccef8', 'request_id': 'req-f040fcae-4c4e-4407-b498-b1615a803d69'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:45.618687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f366d4452d0c4c81b33da1b2c71ccef8', 'request_id': 'req-f040fcae-4c4e-4407-b498-b1615a803d69'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:45.717959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] The token cache is being invalidate because project f366d4452d0c4c81b33da1b2c71ccef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:00:45.718281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:45.718806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f040fcae-4c4e-4407-b498-b1615a803d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:00:45.720192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 750/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:00:45 2022] DELETE /identity/v3/projects/f366d4452d0c4c81b33da1b2c71ccef8 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:00:54.832951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=62 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:00:55.715898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:02:14.984862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c79020-e746-4808-b4bf-181da4036d89 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:14.985497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c79020-e746-4808-b4bf-181da4036d89 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:14.985497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c79020-e746-4808-b4bf-181da4036d89 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:14.986949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 11:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:14.992138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dabe8e-7317-461f-a20e-9e403ce89a4f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:14.992657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dabe8e-7317-461f-a20e-9e403ce89a4f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:14.992772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dabe8e-7317-461f-a20e-9e403ce89a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:15.071594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-78dabe8e-7317-461f-a20e-9e403ce89a4f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:15.072795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-78dabe8e-7317-461f-a20e-9e403ce89a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:15.175840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 11:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:15.330934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c79020-e746-4808-b4bf-181da4036d89 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:15.340764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-58c79020-e746-4808-b4bf-181da4036d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:15.449331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c79020-e746-4808-b4bf-181da4036d89 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:15.455394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d6823-3275-4a36-b4aa-a85135a8e44c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:15.455993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6823-3275-4a36-b4aa-a85135a8e44c service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:15.456274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6823-3275-4a36-b4aa-a85135a8e44c service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:15.456543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d6823-3275-4a36-b4aa-a85135a8e44c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:15.464383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d6823-3275-4a36-b4aa-a85135a8e44c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:15.465950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d6823-3275-4a36-b4aa-a85135a8e44c service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:15.475357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 762/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:16.310471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c22d7-4c7f-4b97-a5b7-941737822e9b None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:16.311083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c22d7-4c7f-4b97-a5b7-941737822e9b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:16.311213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c22d7-4c7f-4b97-a5b7-941737822e9b None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:16.312494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 11:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:16.317274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fe95b-77d8-459f-b6aa-086a208ae3c1 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:16.317687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fe95b-77d8-459f-b6aa-086a208ae3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:16.317798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fe95b-77d8-459f-b6aa-086a208ae3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:16.337360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ea9fe95b-77d8-459f-b6aa-086a208ae3c1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:16.338584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ea9fe95b-77d8-459f-b6aa-086a208ae3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:16.347063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 11:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:16.517755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c22d7-4c7f-4b97-a5b7-941737822e9b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:16.532607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c22d7-4c7f-4b97-a5b7-941737822e9b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:16.537985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a51a51-ee1c-4058-a7a9-cb01137deee4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:16.538613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a51a51-ee1c-4058-a7a9-cb01137deee4 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:16.538890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a51a51-ee1c-4058-a7a9-cb01137deee4 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:16.539155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a51a51-ee1c-4058-a7a9-cb01137deee4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:16.547735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a51a51-ee1c-4058-a7a9-cb01137deee4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:16.549298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a51a51-ee1c-4058-a7a9-cb01137deee4 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:16.558932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 753/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:17.231620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51bc21-5cb8-4363-842b-da60e65a44e3 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:17.231923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51bc21-5cb8-4363-842b-da60e65a44e3 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:17.232768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51bc21-5cb8-4363-842b-da60e65a44e3 None None] PATH_INFO: `/v3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:17.234116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 764/1517] 10.222.0.47 () {54 vars in 925 bytes} [Mon Aug 22 11:02:17 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:02:17.254980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebde00-e0cd-4843-9389-45c5fa1d5a7c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:17.255268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebde00-e0cd-4843-9389-45c5fa1d5a7c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:17.255543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebde00-e0cd-4843-9389-45c5fa1d5a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:17.316234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2eebde00-e0cd-4843-9389-45c5fa1d5a7c None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:17.317458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2eebde00-e0cd-4843-9389-45c5fa1d5a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:17.333670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:02:17.387333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 754/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Aug 22 11:02:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:21.372350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e166d9-d848-4672-b33d-cdb9c5713381 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:21.372852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e166d9-d848-4672-b33d-cdb9c5713381 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:21.372852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e166d9-d848-4672-b33d-cdb9c5713381 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:21.374665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 765/1519] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 22 11:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:21.379777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4dd04-1986-4614-8c04-256e2fe9df9c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:21.380192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4dd04-1986-4614-8c04-256e2fe9df9c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:21.380333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4dd04-1986-4614-8c04-256e2fe9df9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:21.400007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-64c4dd04-1986-4614-8c04-256e2fe9df9c None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:21.401194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-64c4dd04-1986-4614-8c04-256e2fe9df9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:21.409624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 755/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 22 11:02:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:21.416939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e166d9-d848-4672-b33d-cdb9c5713381 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:21.430073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e166d9-d848-4672-b33d-cdb9c5713381 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:21.434907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa485f7d-b121-46f1-9e03-ff7cf20dee81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:21.436112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa485f7d-b121-46f1-9e03-ff7cf20dee81 service placement] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:21.436466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa485f7d-b121-46f1-9e03-ff7cf20dee81 service placement] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:21.436733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa485f7d-b121-46f1-9e03-ff7cf20dee81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:21.445845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa485f7d-b121-46f1-9e03-ff7cf20dee81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:21.447446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa485f7d-b121-46f1-9e03-ff7cf20dee81 service placement] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:21.458559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 766/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:27.784194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7ab64-bd52-4a9f-91df-19212b69f285 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:27.784805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7ab64-bd52-4a9f-91df-19212b69f285 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:27.784805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7ab64-bd52-4a9f-91df-19212b69f285 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:27.786139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 11:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:27.791025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f682b95-fdec-4ab9-9c21-7986342f2beb None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:27.793483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f682b95-fdec-4ab9-9c21-7986342f2beb None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:27.793483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f682b95-fdec-4ab9-9c21-7986342f2beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:27.811910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8f682b95-fdec-4ab9-9c21-7986342f2beb None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:27.813277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8f682b95-fdec-4ab9-9c21-7986342f2beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:27.821116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 11:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:27.975320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7ab64-bd52-4a9f-91df-19212b69f285 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:27.989868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7ab64-bd52-4a9f-91df-19212b69f285 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:27.995279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12d2c4-5a60-48c6-81a7-5cb272f8f5e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:27.995895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12d2c4-5a60-48c6-81a7-5cb272f8f5e9 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:27.996173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12d2c4-5a60-48c6-81a7-5cb272f8f5e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:27.996437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12d2c4-5a60-48c6-81a7-5cb272f8f5e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:28.005547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12d2c4-5a60-48c6-81a7-5cb272f8f5e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:28.007181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f12d2c4-5a60-48c6-81a7-5cb272f8f5e9 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:28.017293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 757/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:02:28.833790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112de2a-d9ea-400a-853e-3ce22fba1ab9 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:28.834376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112de2a-d9ea-400a-853e-3ce22fba1ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:28.834376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112de2a-d9ea-400a-853e-3ce22fba1ab9 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:28.835697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 22 11:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:28.840350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074131a-b03d-4d3e-9dd7-e386610f52a1 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:28.840776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074131a-b03d-4d3e-9dd7-e386610f52a1 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:28.840878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074131a-b03d-4d3e-9dd7-e386610f52a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:28.860917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b074131a-b03d-4d3e-9dd7-e386610f52a1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:28.862120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b074131a-b03d-4d3e-9dd7-e386610f52a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:28.870185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 22 11:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:29.020875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112de2a-d9ea-400a-853e-3ce22fba1ab9 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:29.035561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112de2a-d9ea-400a-853e-3ce22fba1ab9 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:29.041003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac221f-4885-4262-b214-d15c5513f510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:29.041595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac221f-4885-4262-b214-d15c5513f510 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:29.041875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac221f-4885-4262-b214-d15c5513f510 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:29.042149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac221f-4885-4262-b214-d15c5513f510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:29.051169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac221f-4885-4262-b214-d15c5513f510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:29.052740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac221f-4885-4262-b214-d15c5513f510 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:29.063164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 769/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:33.010285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba8f5a-5490-45c2-a35f-4aeca211164e None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:33.010837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba8f5a-5490-45c2-a35f-4aeca211164e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:33.010837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba8f5a-5490-45c2-a35f-4aeca211164e None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:33.012180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 11:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:33.017864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418a5a-ccab-4268-9833-a0a1f95ebd45 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:33.018325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418a5a-ccab-4268-9833-a0a1f95ebd45 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:33.018447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e418a5a-ccab-4268-9833-a0a1f95ebd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:33.038620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-5e418a5a-ccab-4268-9833-a0a1f95ebd45 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:33.039814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5e418a5a-ccab-4268-9833-a0a1f95ebd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:33.047888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 11:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:02:33.341996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba8f5a-5490-45c2-a35f-4aeca211164e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:33.343720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-10ba8f5a-5490-45c2-a35f-4aeca211164e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:33.420219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba8f5a-5490-45c2-a35f-4aeca211164e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:33.423856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abb48d-4c2c-4ab2-a38d-5370cc4eb775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:33.424458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abb48d-4c2c-4ab2-a38d-5370cc4eb775 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:33.424737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abb48d-4c2c-4ab2-a38d-5370cc4eb775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:33.425047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abb48d-4c2c-4ab2-a38d-5370cc4eb775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:33.433712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abb48d-4c2c-4ab2-a38d-5370cc4eb775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:33.435206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abb48d-4c2c-4ab2-a38d-5370cc4eb775 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:33.445315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 760/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:35.331590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1d71-3475-4a6c-9cfc-21f6212a4ac1 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:35.332163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1d71-3475-4a6c-9cfc-21f6212a4ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:35.332163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1d71-3475-4a6c-9cfc-21f6212a4ac1 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:35.333473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 11:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:35.338641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbb779-8141-4b49-a9ba-a1d46dd6ecc4 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:35.339067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbb779-8141-4b49-a9ba-a1d46dd6ecc4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:35.339190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbb779-8141-4b49-a9ba-a1d46dd6ecc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:35.359929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-87cbb779-8141-4b49-a9ba-a1d46dd6ecc4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:35.361132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-87cbb779-8141-4b49-a9ba-a1d46dd6ecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:35.369219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 11:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:35.657185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a1d71-3475-4a6c-9cfc-21f6212a4ac1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:35.668012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a1d71-3475-4a6c-9cfc-21f6212a4ac1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:35.672306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26829d5-7757-4589-b32f-3927dc3deab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:35.672906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26829d5-7757-4589-b32f-3927dc3deab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:35.673184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26829d5-7757-4589-b32f-3927dc3deab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:35.673457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26829d5-7757-4589-b32f-3927dc3deab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:35.682199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26829d5-7757-4589-b32f-3927dc3deab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:35.683656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26829d5-7757-4589-b32f-3927dc3deab4 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:35.693032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 772/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:39.126720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9770ad-cfe9-46b1-8188-b6988941370a None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:39.127256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9770ad-cfe9-46b1-8188-b6988941370a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:39.127256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9770ad-cfe9-46b1-8188-b6988941370a None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:39.128595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 11:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 11:02:39.133716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a53231-3cac-4ea8-bb6d-b0b400119ea7 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:39.134168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a53231-3cac-4ea8-bb6d-b0b400119ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:39.134300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a53231-3cac-4ea8-bb6d-b0b400119ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:39.153710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-65a53231-3cac-4ea8-bb6d-b0b400119ea7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:39.154930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-65a53231-3cac-4ea8-bb6d-b0b400119ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:39.162764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:39.713149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9770ad-cfe9-46b1-8188-b6988941370a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:39.714839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-aa9770ad-cfe9-46b1-8188-b6988941370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:39.778892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:02:39.791798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9770ad-cfe9-46b1-8188-b6988941370a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:39.795470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96c8c5-d45c-4c70-81c0-88140b2468c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:39.796072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96c8c5-d45c-4c70-81c0-88140b2468c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:39.797580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96c8c5-d45c-4c70-81c0-88140b2468c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:39.797860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96c8c5-d45c-4c70-81c0-88140b2468c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:39.806916 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96c8c5-d45c-4c70-81c0-88140b2468c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:39.808374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96c8c5-d45c-4c70-81c0-88140b2468c6 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:39.818372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 763/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:41.737931 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba303a-3999-4e47-8084-c355a79b7b0a None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:41.738778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba303a-3999-4e47-8084-c355a79b7b0a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:41.738778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba303a-3999-4e47-8084-c355a79b7b0a None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:41.740108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 22 11:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:02:41.745342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867f88-eaf2-435b-bc89-96210515af5d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:41.745780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867f88-eaf2-435b-bc89-96210515af5d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:41.745896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c867f88-eaf2-435b-bc89-96210515af5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:41.767097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-1c867f88-eaf2-435b-bc89-96210515af5d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:02:41.775180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1c867f88-eaf2-435b-bc89-96210515af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:41.776241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 22 11:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:02:42.324628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba303a-3999-4e47-8084-c355a79b7b0a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:02:42.326343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6bba303a-3999-4e47-8084-c355a79b7b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:02:42.339838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba303a-3999-4e47-8084-c355a79b7b0a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:02:42.343982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fbdc2-cfd2-4c7f-b088-69f15b709e17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:02:42.344572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fbdc2-cfd2-4c7f-b088-69f15b709e17 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:02:42.344845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fbdc2-cfd2-4c7f-b088-69f15b709e17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:02:42.345113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fbdc2-cfd2-4c7f-b088-69f15b709e17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:02:42.353913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fbdc2-cfd2-4c7f-b088-69f15b709e17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:02:42.355369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71fbdc2-cfd2-4c7f-b088-69f15b709e17 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:02:42.365580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 775/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:02:51.760928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=11 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:02:52.361252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:08.267910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57e9e-dc4e-4001-b1ea-5a637d6fc81b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.269047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57e9e-dc4e-4001-b1ea-5a637d6fc81b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.269700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a57e9e-dc4e-4001-b1ea-5a637d6fc81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.277696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01fd36-ef59-4f94-a321-bb061825a3c8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.278255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01fd36-ef59-4f94-a321-bb061825a3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.278255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01fd36-ef59-4f94-a321-bb061825a3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.300460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-79a57e9e-dc4e-4001-b1ea-5a637d6fc81b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.302514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-79a57e9e-dc4e-4001-b1ea-5a637d6fc81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.307549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6c01fd36-ef59-4f94-a321-bb061825a3c8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.308764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6c01fd36-ef59-4f94-a321-bb061825a3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.317809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 765/1540] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.323427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 776/1541] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.333277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd6dc-3aa1-42cc-b469-a0bbea0949ca None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.333908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd6dc-3aa1-42cc-b469-a0bbea0949ca None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.334033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a033d-5c44-4a8b-b265-a2e5534f45a8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.334317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a033d-5c44-4a8b-b265-a2e5534f45a8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.334594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a033d-5c44-4a8b-b265-a2e5534f45a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.335022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbd6dc-3aa1-42cc-b469-a0bbea0949ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.368803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4b8a033d-5c44-4a8b-b265-a2e5534f45a8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.368803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4b8a033d-5c44-4a8b-b265-a2e5534f45a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.371243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c7dbd6dc-3aa1-42cc-b469-a0bbea0949ca None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.373091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c7dbd6dc-3aa1-42cc-b469-a0bbea0949ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.380636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 777/1542] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.384706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 766/1543] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.385391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a033d-5c44-4a8b-b265-a2e5534f45a8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:08.389092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbd6dc-3aa1-42cc-b469-a0bbea0949ca None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:08.399407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a033d-5c44-4a8b-b265-a2e5534f45a8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:08.405074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbd6dc-3aa1-42cc-b469-a0bbea0949ca None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:08.414947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:08.415581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.415858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.416199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.417373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:08.417672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:08.418720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING py.warnings [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:04:08.418720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: warnings.warn(msg) Aug 22 11:04:08.418720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: Aug 22 11:04:08.419607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156de8d-c2aa-4810-9672-046918c6f064 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:08.422956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea40cda-c402-4267-8d08-56f6ca9dbda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:08.423596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea40cda-c402-4267-8d08-56f6ca9dbda0 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.423877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea40cda-c402-4267-8d08-56f6ca9dbda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.424149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea40cda-c402-4267-8d08-56f6ca9dbda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.424811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea40cda-c402-4267-8d08-56f6ca9dbda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:08.425113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea40cda-c402-4267-8d08-56f6ca9dbda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:08.426757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea40cda-c402-4267-8d08-56f6ca9dbda0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:08.432649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 778/1544] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:08.433914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49e3da-c229-4867-8906-6d2ff13b7941 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.434197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49e3da-c229-4867-8906-6d2ff13b7941 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.434491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49e3da-c229-4867-8906-6d2ff13b7941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.452093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 767/1545] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:04:08.453501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96417178-2a3e-4e2d-bc4c-48a6b64be71a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.453793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96417178-2a3e-4e2d-bc4c-48a6b64be71a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.454073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96417178-2a3e-4e2d-bc4c-48a6b64be71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.460097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-fb49e3da-c229-4867-8906-6d2ff13b7941 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.461307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-fb49e3da-c229-4867-8906-6d2ff13b7941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.474969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 779/1546] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.477443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e220a43-b398-49ed-9376-b1ead5fd2e8f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.477727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e220a43-b398-49ed-9376-b1ead5fd2e8f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.478000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e220a43-b398-49ed-9376-b1ead5fd2e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.499825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-96417178-2a3e-4e2d-bc4c-48a6b64be71a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.500160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7e220a43-b398-49ed-9376-b1ead5fd2e8f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.501041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-96417178-2a3e-4e2d-bc4c-48a6b64be71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.501361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7e220a43-b398-49ed-9376-b1ead5fd2e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.509618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 768/1547] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.510714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 780/1548] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.511025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96417178-2a3e-4e2d-bc4c-48a6b64be71a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:08.512111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e220a43-b398-49ed-9376-b1ead5fd2e8f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:08.521079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96417178-2a3e-4e2d-bc4c-48a6b64be71a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:08.522844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e220a43-b398-49ed-9376-b1ead5fd2e8f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:08.526141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d5dc6-6d8f-4bba-960b-d5fb999df149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:08.526949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5dc6-6d8f-4bba-960b-d5fb999df149 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.527225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5dc6-6d8f-4bba-960b-d5fb999df149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.527488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d5dc6-6d8f-4bba-960b-d5fb999df149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.528034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136ac5-5789-4da9-807b-d8d7a4dd8757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:08.528158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d5dc6-6d8f-4bba-960b-d5fb999df149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:08.528645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136ac5-5789-4da9-807b-d8d7a4dd8757 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.528927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136ac5-5789-4da9-807b-d8d7a4dd8757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.529110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-1b2d5dc6-6d8f-4bba-960b-d5fb999df149 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:04:08.529110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 11:04:08.529110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 11:04:08.529392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136ac5-5789-4da9-807b-d8d7a4dd8757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.529804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21136ac5-5789-4da9-807b-d8d7a4dd8757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:08.530308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2d5dc6-6d8f-4bba-960b-d5fb999df149 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:08.531748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21136ac5-5789-4da9-807b-d8d7a4dd8757 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:08.647675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 769/1549] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:08.649156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18f02-6b4f-4032-b59e-40ffe70ad1a8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.649448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18f02-6b4f-4032-b59e-40ffe70ad1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.649725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18f02-6b4f-4032-b59e-40ffe70ad1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.660542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 781/1550] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:08.660542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebf88e-498b-41d8-b9f9-0163bb04d725 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.660542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebf88e-498b-41d8-b9f9-0163bb04d725 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.660542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebf88e-498b-41d8-b9f9-0163bb04d725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.673471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-35d18f02-6b4f-4032-b59e-40ffe70ad1a8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.674742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-35d18f02-6b4f-4032-b59e-40ffe70ad1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.679639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-80ebf88e-498b-41d8-b9f9-0163bb04d725 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.680827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-80ebf88e-498b-41d8-b9f9-0163bb04d725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.741897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 770/1551] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.743192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f0d0-daeb-4526-879e-f1fa2760d388 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.743479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f0d0-daeb-4526-879e-f1fa2760d388 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.743747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f0d0-daeb-4526-879e-f1fa2760d388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.753581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 782/1552] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.755149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232fa1a-90d5-4fbf-8ef4-c5000d182e5a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.755611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232fa1a-90d5-4fbf-8ef4-c5000d182e5a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.755885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232fa1a-90d5-4fbf-8ef4-c5000d182e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.764137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0a87f0d0-daeb-4526-879e-f1fa2760d388 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.765331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0a87f0d0-daeb-4526-879e-f1fa2760d388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.773332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 771/1553] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.774527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddd47-b7a2-4d4c-90bd-5688d31eb5ad None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.774806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddd47-b7a2-4d4c-90bd-5688d31eb5ad None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.775075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ddd47-b7a2-4d4c-90bd-5688d31eb5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.775282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f232fa1a-90d5-4fbf-8ef4-c5000d182e5a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.776587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f232fa1a-90d5-4fbf-8ef4-c5000d182e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.786404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 783/1554] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.787641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c467f-a228-4f6f-8090-79f408bd606c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.787921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c467f-a228-4f6f-8090-79f408bd606c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.788188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c467f-a228-4f6f-8090-79f408bd606c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.797153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-408ddd47-b7a2-4d4c-90bd-5688d31eb5ad None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.798402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-408ddd47-b7a2-4d4c-90bd-5688d31eb5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.806200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 772/1555] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.807184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddd47-b7a2-4d4c-90bd-5688d31eb5ad None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:08.813873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8d7c467f-a228-4f6f-8090-79f408bd606c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.815156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8d7c467f-a228-4f6f-8090-79f408bd606c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.816521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ddd47-b7a2-4d4c-90bd-5688d31eb5ad None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:08.821073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03127-f14e-4c84-9011-32f1b06b2c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:08.821628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03127-f14e-4c84-9011-32f1b06b2c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.821931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03127-f14e-4c84-9011-32f1b06b2c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.822203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03127-f14e-4c84-9011-32f1b06b2c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.823388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03127-f14e-4c84-9011-32f1b06b2c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:08.823916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 784/1556] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.824632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c467f-a228-4f6f-8090-79f408bd606c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:08.825422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03127-f14e-4c84-9011-32f1b06b2c3f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:08.834149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c467f-a228-4f6f-8090-79f408bd606c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:08.839014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:08.839608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.839923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.840194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.840815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:08.841841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING py.warnings [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:04:08.841841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: warnings.warn(msg) Aug 22 11:04:08.841841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: Aug 22 11:04:08.843070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:08.926660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 773/1557] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:08.928364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b630c-9971-4575-b840-1fa36fd5fcf6 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.928364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b630c-9971-4575-b840-1fa36fd5fcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.928582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b630c-9971-4575-b840-1fa36fd5fcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.949828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-702b630c-9971-4575-b840-1fa36fd5fcf6 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.951121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-702b630c-9971-4575-b840-1fa36fd5fcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.956304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 785/1558] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:08.956304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:08.961090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 774/1559] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:08.962323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa074a-dbf0-42b6-adc0-7428f2ec3334 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.962607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa074a-dbf0-42b6-adc0-7428f2ec3334 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.962892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa074a-dbf0-42b6-adc0-7428f2ec3334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.965852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14298d1-5f3e-4137-a4d7-b182ab480955 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:08.970553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bced7-5c36-41c4-8fae-c262fdfbe863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:08.971926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bced7-5c36-41c4-8fae-c262fdfbe863 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.971926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bced7-5c36-41c4-8fae-c262fdfbe863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.971926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bced7-5c36-41c4-8fae-c262fdfbe863 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:08.972295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bced7-5c36-41c4-8fae-c262fdfbe863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:08.972590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bced7-5c36-41c4-8fae-c262fdfbe863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:08.974192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bced7-5c36-41c4-8fae-c262fdfbe863 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:08.983048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3caa074a-dbf0-42b6-adc0-7428f2ec3334 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:08.986600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3caa074a-dbf0-42b6-adc0-7428f2ec3334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:08.986983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 786/1560] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:08.988171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e70fd-28d3-49dc-b5ae-bee8b02b9919 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:08.988450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e70fd-28d3-49dc-b5ae-bee8b02b9919 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:08.988719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e70fd-28d3-49dc-b5ae-bee8b02b9919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.000515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 775/1561] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:09.001339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa074a-dbf0-42b6-adc0-7428f2ec3334 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.008860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-318e70fd-28d3-49dc-b5ae-bee8b02b9919 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:09.010108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-318e70fd-28d3-49dc-b5ae-bee8b02b9919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:09.010874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa074a-dbf0-42b6-adc0-7428f2ec3334 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.015369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.016032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.016256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.016522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.017156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:09.017454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.017890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 787/1562] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:09.018795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e70fd-28d3-49dc-b5ae-bee8b02b9919 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.019074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.030313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e70fd-28d3-49dc-b5ae-bee8b02b9919 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.036032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 776/1563] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:09.036032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.040780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.041774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e7669-e8a2-4129-bc9e-664228a21428 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.041990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.042214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.042504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.043140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:09.043440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.045039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.046438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.047140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.047419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.047689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.048316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.049859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.058498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 788/1564] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:09.059170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.063936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 777/1565] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:09.063936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.069633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc58188-72cf-4429-a4b0-bb029ddd543a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.072810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7203e4-7d5a-4bd0-a9a3-12fcb1f9bde9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.074164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.074800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.075079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.075348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.075975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.077376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.077597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.077955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.078256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.078602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.079234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:09.079821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.081078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.087801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 789/1566] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:09.089158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.093645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 778/1567] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:09.094646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.098900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850ac63-0546-49f5-ae5d-4aec725784de admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.103372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.104647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.104917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.105179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.105792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.106442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5eca4e-10d0-4f0c-a486-b879bccb98f0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.107688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.111045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.111920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.112200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.112475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.113416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:09.113416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.115035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.122086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:09.127453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 779/1568] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:09.128271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.133056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:09.138460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeece6-a5fe-4277-840f-170762bae8f0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.143000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.143572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.143846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.144114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.144739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:09.145033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.146656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.158224 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 780/1569] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:09.158995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.167970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21491a-1127-4fca-ac07-3abcdd7f3d65 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.172495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.173059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.173350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.173607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.174198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.176100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.184150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 790/1570] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:09.184150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.257323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b0961-d5ab-42a2-9480-5b505f3b7646 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.262580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.263193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.263477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.263757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.264409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.266004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.276748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 791/1571] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:09.277878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.284297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 781/1572] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:09.284297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.289699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751289c6-82f5-4fa1-bfec-9c6c7d881413 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.344914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ea3a2-f84a-4fc6-9a98-887b2ccef50d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.349096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.349610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.349892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.350177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.350302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.350479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.350590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.350832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.351099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.351445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.353033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.353141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.364410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 782/1573] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:09.365357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.376156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e4e08-3c65-453e-9fef-10a24a0d6291 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.380783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.381382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.381660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.382083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.382691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.387591 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.466400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 792/1574] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:09.467287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.490154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 783/1575] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:09.490154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.528735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b05cd8-bf7f-4584-b482-b745ae838728 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.547558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:09.576461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb67b6f-ea1e-4ff3-9416-331f2cfbcabc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.584423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.585045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.585322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.586384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.586384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.586907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.587515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.587794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.588712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.588831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.589333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.591256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.675619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 784/1576] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:09.676416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.686616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 793/1577] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/projects => generated 405 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:09.687506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.707609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:09.739968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08692-bd0b-401b-984f-4af1063376b8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.750415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f79aa7-35ad-4810-8bdb-3403bc491038 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.755340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.756170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.756792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.756792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] PATH_INFO: `/v3/projects/793621196ef245588f21b7982fe651e1/users/7acea30e695341c8a1d83f386629a99b/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.772363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=793621196ef245588f21b7982fe651e1, user_id=7acea30e695341c8a1d83f386629a99b, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.776220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.798582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.798793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.798935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.799195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.799808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.801703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.876629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 794/1578] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:09 2022] PUT /identity/v3/projects/793621196ef245588f21b7982fe651e1/users/7acea30e695341c8a1d83f386629a99b/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:09.877456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.883309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 785/1579] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:09.883309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:09.894462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d863d3e-3fc4-4eab-9e6d-73be57ab276d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.941397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c348d-a370-4203-b7ff-539ffb6baf87 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:09.946051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.946685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.946962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.947240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.948483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.950407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:09.951052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:09.951724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:09.952007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:09.952285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] PATH_INFO: `/v3/projects/629e8810bbcb4623bcd6b6aee8f7dbb9/users/d13b7df898df4d238ea290fab8edc4a7/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:09.966856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629e8810bbcb4623bcd6b6aee8f7dbb9, user_id=d13b7df898df4d238ea290fab8edc4a7, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:09.967902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:04:09.967902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 11:04:09.967902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 11:04:09.971047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.059508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 795/1580] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:09 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:10.060371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.069928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b1f1e-bdda-4455-942e-6a1faf8043c1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.076666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 786/1581] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:09 2022] PUT /identity/v3/projects/629e8810bbcb4623bcd6b6aee8f7dbb9/users/d13b7df898df4d238ea290fab8edc4a7/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:10.080551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.088413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:10.126530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:10.129951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.130625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.130932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.131207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.131829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.133735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.143956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092178a-8481-4bdf-a69b-98dda78a6fb3 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.148628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.149241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.149526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.149801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.150477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.152436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.233825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 796/1582] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:10 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:10.233825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.237246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 787/1583] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:10 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:10.238518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.242027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e580-65b9-4a0b-9412-fe15a98006c4 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.247700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7e90e-e193-4c9e-b690-3335bfce0102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.248229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e90e-e193-4c9e-b690-3335bfce0102 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.248539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e90e-e193-4c9e-b690-3335bfce0102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.248816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7e90e-e193-4c9e-b690-3335bfce0102 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.249436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7e90e-e193-4c9e-b690-3335bfce0102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.249785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca8c81-f201-45f8-ad70-ed9206f73c21 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.251720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7e90e-e193-4c9e-b690-3335bfce0102 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.257503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.257503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.257503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.257503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.257868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.258317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.334362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 797/1584] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:10 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:10.335164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 788/1585] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:10 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:10.335164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.335615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac3bc1-a219-411a-9b93-856bd340aba3 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.335615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac3bc1-a219-411a-9b93-856bd340aba3 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.335615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac3bc1-a219-411a-9b93-856bd340aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.345003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c5eaa-b018-4c24-b3ff-faf8d0273814 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.350160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad442d-6f7c-4e22-ae95-0c3305e33bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.350781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad442d-6f7c-4e22-ae95-0c3305e33bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.351060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad442d-6f7c-4e22-ae95-0c3305e33bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.351327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad442d-6f7c-4e22-ae95-0c3305e33bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.351967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad442d-6f7c-4e22-ae95-0c3305e33bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.352556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c7ac3bc1-a219-411a-9b93-856bd340aba3 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:10.353557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad442d-6f7c-4e22-ae95-0c3305e33bb7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.353823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c7ac3bc1-a219-411a-9b93-856bd340aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:10.362060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 798/1586] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:10.365471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac3bc1-a219-411a-9b93-856bd340aba3 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.365571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 789/1587] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.365837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3aff2-13bc-479f-92c3-c10483a09ff7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.366260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3aff2-13bc-479f-92c3-c10483a09ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.366529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3aff2-13bc-479f-92c3-c10483a09ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.372386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac3bc1-a219-411a-9b93-856bd340aba3 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.377401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.377999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.378305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.378578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.379223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.380812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.384802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0ad3aff2-13bc-479f-92c3-c10483a09ff7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:10.386603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0ad3aff2-13bc-479f-92c3-c10483a09ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:10.391570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 799/1588] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.392418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.397216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 790/1589] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:10.398196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3aff2-13bc-479f-92c3-c10483a09ff7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.403351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791da9e1-96a0-4664-b918-de3094b1a69e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.408319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.408487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3aff2-13bc-479f-92c3-c10483a09ff7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.408925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.409205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.409475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.410119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.411721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.413390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.413997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.414308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.414581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.415229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.416902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.422107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 800/1590] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.425830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.426922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 791/1591] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.427709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.435726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21de9a-0133-4ac6-afb2-70f515f4d28b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.438638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bc951-7773-4993-8405-1371e1659790 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.440266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.440903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.441180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.441451 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.442102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.443242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.443740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.443872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.444081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.444348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.444986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.446571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.454733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 801/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.455501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.456838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 792/1593] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.457692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.467495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62c17-b1da-41aa-977d-e4f6eb44e7cf admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.472138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.472726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.473004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.473274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.473906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.475522 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.485953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 802/1594] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.487024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.497347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f4559-7f8a-47f1-8f7f-cb47fbf97728 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.502738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.503312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.503907 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.504182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.504965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.506564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.516287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc2421-7675-40d3-8f48-df8768e0548c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.517260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 803/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.517951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.521570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:10.523048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.523683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.523963 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.524238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.533998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.534134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af321bd2-4e25-46f2-95d8-f22fd6ca6df0 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.535459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.541248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.541870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.542440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.542722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.545441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 793/1596] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:10.546063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.552850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.554215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.555265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46dfdb-5574-4d4a-a56f-6cd106482144 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.561437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.562771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.563073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.563436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.564035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.565274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 804/1597] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:10.565942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.566300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.576390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737939fc-6d6b-4bf3-a3af-eede3093d7d1 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.578210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 794/1598] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.581548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.581673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.582344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.582683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.583096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.584193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.585737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.594570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5e944-b5d9-4b59-abf7-22a597eab9f9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.598372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.598989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.599271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.599587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.600211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.605629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 805/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.611699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.611858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.616393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592d58-4980-4965-926f-06ad90275575 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.616833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 795/1600] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.617881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.621650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.622352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.622661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.622930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.623572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.625162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.628268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a3658-68b3-4803-9996-f9d3d3e82cbc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.633022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.633691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.633967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.639492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] PATH_INFO: `/v3/projects/b2fee426407b48138c95541c02b38637/users/6e9ddaf55d0a48219dbe517c7ff4aa32/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.640412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 806/1601] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:10.641390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.650475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fee426407b48138c95541c02b38637, user_id=6e9ddaf55d0a48219dbe517c7ff4aa32, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.652280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6273e-c6b1-418f-bea5-ce2e221594c5 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.654281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.657541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.658299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.658586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.658864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] PATH_INFO: `/v3/projects/7d88aee438994d1b96443472f132ea77/users/fd597b7fd9414dcab382940be9c14106/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.679196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d88aee438994d1b96443472f132ea77, user_id=fd597b7fd9414dcab382940be9c14106, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.682643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.756258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 796/1602] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:10 2022] PUT /identity/v3/projects/b2fee426407b48138c95541c02b38637/users/6e9ddaf55d0a48219dbe517c7ff4aa32/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:10.756847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.791588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 807/1603] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:10 2022] PUT /identity/v3/projects/7d88aee438994d1b96443472f132ea77/users/fd597b7fd9414dcab382940be9c14106/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:10.791588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:10.797777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:10.827491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d457fb-eee7-4414-b35b-e259ff2c7969 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.842459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:10.890385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d445994-1ebb-4098-853d-66cdf76caf18 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:10.899769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.900591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.901802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.903997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] PATH_INFO: `/v3/projects/2cb7651353884366b2ca72a7e272bdc0/users/7c6fee0b956e41a7b2d9b914043633ea/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.925227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb7651353884366b2ca72a7e272bdc0, user_id=7c6fee0b956e41a7b2d9b914043633ea, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.929870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:10.934496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:10.935270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:10.935552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:10.935825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] PATH_INFO: `/v3/projects/d809822ae9ba4c248d5941746efff780/users/a1cdc3bfa3e046fa863131d7ff27d4a1/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:10.953738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d809822ae9ba4c248d5941746efff780, user_id=a1cdc3bfa3e046fa863131d7ff27d4a1, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:10.957334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:11.013815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 808/1604] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:10 2022] PUT /identity/v3/projects/2cb7651353884366b2ca72a7e272bdc0/users/7c6fee0b956e41a7b2d9b914043633ea/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:11.013815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:11.049504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 797/1605] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:10 2022] PUT /identity/v3/projects/d809822ae9ba4c248d5941746efff780/users/a1cdc3bfa3e046fa863131d7ff27d4a1/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:11.050358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:11.105777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893a496-53a3-40a4-b9e3-03387a880d32 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:11.126799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e46dd3-35d8-4153-ac06-750da4aa879e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:11.132579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171828e-6942-4afb-a696-9f5bf01d7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:11.133354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171828e-6942-4afb-a696-9f5bf01d7fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.133833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171828e-6942-4afb-a696-9f5bf01d7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.134186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171828e-6942-4afb-a696-9f5bf01d7fd6 admin admin] PATH_INFO: `/v3/projects/3c1502477fac4fb5aa2edcaf77cef9c2/users/106dae758b6d41c4935e48e323b73ffb/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.154092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171828e-6942-4afb-a696-9f5bf01d7fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1502477fac4fb5aa2edcaf77cef9c2, user_id=106dae758b6d41c4935e48e323b73ffb, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:11.156873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171828e-6942-4afb-a696-9f5bf01d7fd6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:11.183781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e738fa-e62c-4359-8e66-f25d37d5aaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:11.184441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e738fa-e62c-4359-8e66-f25d37d5aaa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.184735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e738fa-e62c-4359-8e66-f25d37d5aaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.184983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e738fa-e62c-4359-8e66-f25d37d5aaa4 admin admin] PATH_INFO: `/v3/projects/6af25d4a38fa43c496a92b6c0c66fb73/users/ebc4e3ad582240f582721525229b7cb5/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.200872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e738fa-e62c-4359-8e66-f25d37d5aaa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af25d4a38fa43c496a92b6c0c66fb73, user_id=ebc4e3ad582240f582721525229b7cb5, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:11.203579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e738fa-e62c-4359-8e66-f25d37d5aaa4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:11.279532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 798/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:11 2022] PUT /identity/v3/projects/3c1502477fac4fb5aa2edcaf77cef9c2/users/106dae758b6d41c4935e48e323b73ffb/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:11.282024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f125ef3-0cbc-4a9e-913b-31192184eb25 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.282024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f125ef3-0cbc-4a9e-913b-31192184eb25 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.282024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f125ef3-0cbc-4a9e-913b-31192184eb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.282988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 809/1607] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:11 2022] PUT /identity/v3/projects/6af25d4a38fa43c496a92b6c0c66fb73/users/ebc4e3ad582240f582721525229b7cb5/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:11.284526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0dba-8a1f-4fbd-90a4-1783aef49140 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.285007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0dba-8a1f-4fbd-90a4-1783aef49140 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.285689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0dba-8a1f-4fbd-90a4-1783aef49140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.337728 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3f125ef3-0cbc-4a9e-913b-31192184eb25 None None] MFA Rules not processed for user `6e9ddaf55d0a48219dbe517c7ff4aa32`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.339297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3f125ef3-0cbc-4a9e-913b-31192184eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.339976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ddec0dba-8a1f-4fbd-90a4-1783aef49140 None None] MFA Rules not processed for user `fd597b7fd9414dcab382940be9c14106`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.341668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ddec0dba-8a1f-4fbd-90a4-1783aef49140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.379407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:11.384830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:11.421474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 799/1609] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.421971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 810/1609] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.422918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be368b5-1330-442f-8dd8-1b25e958dbcc None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.423211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be368b5-1330-442f-8dd8-1b25e958dbcc None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.423489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be368b5-1330-442f-8dd8-1b25e958dbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.424121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b71cf7-5fa9-4310-a731-6ad14c7f77de None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.424654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b71cf7-5fa9-4310-a731-6ad14c7f77de None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.425145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b71cf7-5fa9-4310-a731-6ad14c7f77de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.451665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0be368b5-1330-442f-8dd8-1b25e958dbcc None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.454807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0be368b5-1330-442f-8dd8-1b25e958dbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.460054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-92b71cf7-5fa9-4310-a731-6ad14c7f77de None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.462011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-92b71cf7-5fa9-4310-a731-6ad14c7f77de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.539596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 811/1610] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.541108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7ad69-0cc7-4ac6-ae86-144ffc3ab08a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.541460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7ad69-0cc7-4ac6-ae86-144ffc3ab08a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.541729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7ad69-0cc7-4ac6-ae86-144ffc3ab08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.547819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 800/1611] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.549379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe750d-f561-4329-95ef-f897d59e60c0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.549801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe750d-f561-4329-95ef-f897d59e60c0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.550172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe750d-f561-4329-95ef-f897d59e60c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.578512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-3ff7ad69-0cc7-4ac6-ae86-144ffc3ab08a None None] MFA Rules not processed for user `106dae758b6d41c4935e48e323b73ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.580185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3ff7ad69-0cc7-4ac6-ae86-144ffc3ab08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.592862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-17fe750d-f561-4329-95ef-f897d59e60c0 None None] MFA Rules not processed for user `ebc4e3ad582240f582721525229b7cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.595774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-17fe750d-f561-4329-95ef-f897d59e60c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.654386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 812/1612] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.655631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e804a-cfd1-4190-a2f9-7b154077791b None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.655925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e804a-cfd1-4190-a2f9-7b154077791b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.656197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e804a-cfd1-4190-a2f9-7b154077791b None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.657432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 813/1613] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 22 11:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:11.658845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae8fde-95c9-4d55-a74a-e33b8f86cad9 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.659194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae8fde-95c9-4d55-a74a-e33b8f86cad9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.659467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae8fde-95c9-4d55-a74a-e33b8f86cad9 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.661039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 814/1614] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 22 11:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:11.661750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae8fde-95c9-4d55-a74a-e33b8f86cad9 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:11.674122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 801/1615] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.675115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe750d-f561-4329-95ef-f897d59e60c0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:11.721562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae8fde-95c9-4d55-a74a-e33b8f86cad9 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:11.725553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:11.726340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.726621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.726900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.737141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:11.738486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe750d-f561-4329-95ef-f897d59e60c0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:11.738607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:11.743143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef48076-19b1-4005-8478-ed79505a84d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:11.744027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef48076-19b1-4005-8478-ed79505a84d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.744311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef48076-19b1-4005-8478-ed79505a84d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.744773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef48076-19b1-4005-8478-ed79505a84d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.749326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 815/1616] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:11.753711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:11.759840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8868c5-d38f-449b-a1ec-52ada4303867 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:11.766939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef48076-19b1-4005-8478-ed79505a84d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:11.767231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:11.768078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.768578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.769039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.769620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef48076-19b1-4005-8478-ed79505a84d1 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:11.770120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:11.780315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:11.781559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:11.800329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 802/1617] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:11.802107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd863ed-0172-42ca-85d9-bc81f41a9e40 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.803091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd863ed-0172-42ca-85d9-bc81f41a9e40 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.803091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd863ed-0172-42ca-85d9-bc81f41a9e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.827783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:11.848226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6dd863ed-0172-42ca-85d9-bc81f41a9e40 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.849471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6dd863ed-0172-42ca-85d9-bc81f41a9e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.923404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 803/1618] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.925617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb365ca-3c1a-4151-b174-beb0288cbb08 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:11.925962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb365ca-3c1a-4151-b174-beb0288cbb08 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:11.926195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb365ca-3c1a-4151-b174-beb0288cbb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:11.950559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9fb365ca-3c1a-4151-b174-beb0288cbb08 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:11.951845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9fb365ca-3c1a-4151-b174-beb0288cbb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:11.960185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 816/1619] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/projects => generated 405 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:11.961142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:11.981911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 804/1620] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:11.983103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb365ca-3c1a-4151-b174-beb0288cbb08 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:11.999722 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb365ca-3c1a-4151-b174-beb0288cbb08 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.070204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011986a0-8c48-49bd-bbc1-e69e93d458e9 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.082474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.084780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.085284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.085562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.091222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.093281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.130043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.130871 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.131217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.131638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.199904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.201593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.212328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 805/1621] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:12.213915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:12.247682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6a051-7177-495a-a272-bd8370142b0b service nova] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.248299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 817/1622] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:11 2022] POST /identity/v3/projects => generated 387 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:12.253151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:12.403019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.404392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.404674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.404938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.405563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.407517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.426518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9c9b7-b856-4279-9798-db3b83e31fc1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.433312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.433869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.434347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.434843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.543675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.545662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.569823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 818/1623] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:12.569823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:12.570047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 806/1624] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:12 2022] POST /identity/v3/users => generated 309 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:12.570047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:12.572420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb823e8-8e00-407f-9c15-ebecb4e663fd service nova] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.578845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d781e2c-4359-4291-bf46-1520c18b7d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.580397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d781e2c-4359-4291-bf46-1520c18b7d60 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.580686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d781e2c-4359-4291-bf46-1520c18b7d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.580961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d781e2c-4359-4291-bf46-1520c18b7d60 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.581694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d781e2c-4359-4291-bf46-1520c18b7d60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.584331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d781e2c-4359-4291-bf46-1520c18b7d60 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.590156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefef486-6d7e-4bfd-a064-148b3a5a3baf admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.599341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:12.610881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.611779 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.612057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.612331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.612976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.614618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.636447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 807/1625] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:04:12.640332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:12.660350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794de3b-0d33-419a-8e71-5eb71b548321 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.675425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.676159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.676444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.676771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] PATH_INFO: `/v3/projects/9db7cc628b80407dbaf6156934a2e95c/users/aad8451ccc504648807b986b9ee52549/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.679554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:12.705213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db7cc628b80407dbaf6156934a2e95c, user_id=aad8451ccc504648807b986b9ee52549, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.707947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.733019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 819/1626] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:12 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:12.735082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c669a-35a1-48b5-8656-fe15cfa70e58 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.735539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c669a-35a1-48b5-8656-fe15cfa70e58 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.735948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c669a-35a1-48b5-8656-fe15cfa70e58 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.745601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 820/1627] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 22 11:04:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:12.746938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c669a-35a1-48b5-8656-fe15cfa70e58 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:12.766696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c669a-35a1-48b5-8656-fe15cfa70e58 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.774117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692e780-5d5d-4965-9415-33fc2a9e6098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.776725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e780-5d5d-4965-9415-33fc2a9e6098 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.777443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e780-5d5d-4965-9415-33fc2a9e6098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.778715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e780-5d5d-4965-9415-33fc2a9e6098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.779688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692e780-5d5d-4965-9415-33fc2a9e6098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:12.782925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692e780-5d5d-4965-9415-33fc2a9e6098 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:12.801107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 821/1628] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:12.802665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a69a4-4a37-4a80-8e65-77084bd1cd84 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.803565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a69a4-4a37-4a80-8e65-77084bd1cd84 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.803911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a69a4-4a37-4a80-8e65-77084bd1cd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:12.826840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 808/1629] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:12 2022] PUT /identity/v3/projects/9db7cc628b80407dbaf6156934a2e95c/users/aad8451ccc504648807b986b9ee52549/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:12.827632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:12.857174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d64a69a4-4a37-4a80-8e65-77084bd1cd84 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:12.857174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d64a69a4-4a37-4a80-8e65-77084bd1cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:12.958834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17257b-c051-41e7-8026-1ad467571456 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:12.974643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff67a9-b7f0-4c7b-826c-93ad59abd069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:12.975718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff67a9-b7f0-4c7b-826c-93ad59abd069 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:12.976123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff67a9-b7f0-4c7b-826c-93ad59abd069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:12.976706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff67a9-b7f0-4c7b-826c-93ad59abd069 admin admin] PATH_INFO: `/v3/projects/a6b703bcc49f4debbfc319188b28f13b/users/e3e57ecd8c5d49619298134f2b103181/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.022690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff67a9-b7f0-4c7b-826c-93ad59abd069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b703bcc49f4debbfc319188b28f13b, user_id=e3e57ecd8c5d49619298134f2b103181, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:13.029481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff67a9-b7f0-4c7b-826c-93ad59abd069 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:13.068863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 822/1630] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:04:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:13.070289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a69a4-4a37-4a80-8e65-77084bd1cd84 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:13.084523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a69a4-4a37-4a80-8e65-77084bd1cd84 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:13.093172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:13.093959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.099326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.099954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.107395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:13.118805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 809/1631] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:12 2022] PUT /identity/v3/projects/a6b703bcc49f4debbfc319188b28f13b/users/e3e57ecd8c5d49619298134f2b103181/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:13.121855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517593-df0d-468a-a3a5-db3b2881a241 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.122578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517593-df0d-468a-a3a5-db3b2881a241 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.123652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517593-df0d-468a-a3a5-db3b2881a241 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.125083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 810/1632] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 22 11:04:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:13.125453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:13.128345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517593-df0d-468a-a3a5-db3b2881a241 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:13.148247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 823/1633] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:13.154747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:13.179240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71811aa5-db9a-4baa-9681-66c6ba4cb052 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:13.288618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517593-df0d-468a-a3a5-db3b2881a241 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:13.298082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:13.299945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.303784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.303784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.314650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d074cd-c35a-4d90-94a9-29ae0c02c121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:13.315523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d074cd-c35a-4d90-94a9-29ae0c02c121 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.315523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d074cd-c35a-4d90-94a9-29ae0c02c121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.315742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d074cd-c35a-4d90-94a9-29ae0c02c121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.316504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d074cd-c35a-4d90-94a9-29ae0c02c121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:13.318123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d074cd-c35a-4d90-94a9-29ae0c02c121 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:13.325207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:13.333519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 824/1634] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:13.334850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21c92-e236-4166-a792-6b976e760918 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.335137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21c92-e236-4166-a792-6b976e760918 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.335468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a21c92-e236-4166-a792-6b976e760918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.365009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b4a21c92-e236-4166-a792-6b976e760918 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:13.366485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b4a21c92-e236-4166-a792-6b976e760918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:13.453643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:13.466798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:13.470214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 825/1635] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:13.471500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a21c92-e236-4166-a792-6b976e760918 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:13.472314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:13.488676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a21c92-e236-4166-a792-6b976e760918 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:13.495731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:13.496849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.497397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.499103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] PATH_INFO: `/v3/projects/9db7cc628b80407dbaf6156934a2e95c/users/aad8451ccc504648807b986b9ee52549/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.500623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db7cc628b80407dbaf6156934a2e95c, user_id=aad8451ccc504648807b986b9ee52549, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:13.501907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 811/1636] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:13.502808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:13.506666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:13.533033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842083df-2d48-4e8d-b559-e567ab26cfd8 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:13.538523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 826/1637] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:13 2022] PUT /identity/v3/projects/9db7cc628b80407dbaf6156934a2e95c/users/aad8451ccc504648807b986b9ee52549/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:13.539752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:13.700430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589248ad-c509-46d6-932c-37853738c906 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:13.702273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bf39f-ce7b-43ea-862f-0c0d20609621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:13.703079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf39f-ce7b-43ea-862f-0c0d20609621 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.703357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf39f-ce7b-43ea-862f-0c0d20609621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.703628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf39f-ce7b-43ea-862f-0c0d20609621 admin admin] PATH_INFO: `/v3/projects/a6b703bcc49f4debbfc319188b28f13b/users/e3e57ecd8c5d49619298134f2b103181/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.708818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bf39f-ce7b-43ea-862f-0c0d20609621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b703bcc49f4debbfc319188b28f13b, user_id=e3e57ecd8c5d49619298134f2b103181, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:13.712155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc66c-a982-43e4-be38-13fa819e695f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:13.712793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc66c-a982-43e4-be38-13fa819e695f service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.713598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc66c-a982-43e4-be38-13fa819e695f service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.715684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bf39f-ce7b-43ea-862f-0c0d20609621 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:13.718506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc66c-a982-43e4-be38-13fa819e695f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.743658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 812/1638] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:13 2022] PUT /identity/v3/projects/a6b703bcc49f4debbfc319188b28f13b/users/e3e57ecd8c5d49619298134f2b103181/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:13.747858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1dcaa-dbc3-4331-8757-7fda77cc186f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.747858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1dcaa-dbc3-4331-8757-7fda77cc186f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.757293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1dcaa-dbc3-4331-8757-7fda77cc186f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.882975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-1ae1dcaa-dbc3-4331-8757-7fda77cc186f None None] MFA Rules not processed for user `aad8451ccc504648807b986b9ee52549`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:13.882975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1ae1dcaa-dbc3-4331-8757-7fda77cc186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:13.888524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc66c-a982-43e4-be38-13fa819e695f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:13.888524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc66c-a982-43e4-be38-13fa819e695f service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:13.895282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 827/1639] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:13.897978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a7a9-b7b0-4e33-9597-42428642a365 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:13.899880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a7a9-b7b0-4e33-9597-42428642a365 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:13.900485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a7a9-b7b0-4e33-9597-42428642a365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:13.967065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-93c2a7a9-b7b0-4e33-9597-42428642a365 None None] MFA Rules not processed for user `e3e57ecd8c5d49619298134f2b103181`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:13.969215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-93c2a7a9-b7b0-4e33-9597-42428642a365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:14.094708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 813/1640] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:14.107487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1dcaa-dbc3-4331-8757-7fda77cc186f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:14.179664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 828/1641] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:14.219339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1dcaa-dbc3-4331-8757-7fda77cc186f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:14.225965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:14.226771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:14.227112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:14.227477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:14.239805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:14.242423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:14.266923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 814/1642] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:14 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:14.560508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2a7a9-b7b0-4e33-9597-42428642a365 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:14.560508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2a7a9-b7b0-4e33-9597-42428642a365 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:14.560508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:14.560508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:14.575212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:14.575212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:14.575212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:14.575212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:14.575212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 829/1643] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:14.622817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:14.726432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:14.748002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:14.831155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37c1dc-8154-4d82-ae5b-9f88d7bab065 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:14.839991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:14.840674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:14.840954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:14.841225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] PATH_INFO: `/v3/users/ebc4e3ad582240f582721525229b7cb5` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:14.848687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc4e3ad582240f582721525229b7cb5)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:14.849748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: WARNING py.warnings [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:04:14.849748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: warnings.warn(msg) Aug 22 11:04:14.849748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: Aug 22 11:04:14.872844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:14.967695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:14.991893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80190332-c92b-4b46-9aa4-7ea530d9703e service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:15.014491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:15.015216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:15.015540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:15.015869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] PATH_INFO: `/v3/users/fd597b7fd9414dcab382940be9c14106` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:15.017374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd597b7fd9414dcab382940be9c14106)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:15.026174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:15.257178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc4e3ad582240f582721525229b7cb5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.261315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc4e3ad582240f582721525229b7cb5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.261315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc4e3ad582240f582721525229b7cb5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.313050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd597b7fd9414dcab382940be9c14106'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.314625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd597b7fd9414dcab382940be9c14106'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.318556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd597b7fd9414dcab382940be9c14106'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.338349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd597b7fd9414dcab382940be9c14106'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.373035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc4e3ad582240f582721525229b7cb5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.393631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc4e3ad582240f582721525229b7cb5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.449762 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 815/1644] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:14 2022] DELETE /identity/v3/users/ebc4e3ad582240f582721525229b7cb5 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:15.451015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:15.453413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:15.525644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8457dccb-0b14-418e-bcb1-2357cd982662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd597b7fd9414dcab382940be9c14106'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:15.579831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 830/1645] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:14 2022] DELETE /identity/v3/users/fd597b7fd9414dcab382940be9c14106 => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:15.589429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf2d12-2954-413d-a295-45ce38d99d2b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:15.589861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf2d12-2954-413d-a295-45ce38d99d2b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:15.590311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf2d12-2954-413d-a295-45ce38d99d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:15.597977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e51d8-db1a-4699-b883-f0f07fbc3072 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:15.640685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6dcf2d12-2954-413d-a295-45ce38d99d2b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:15.642094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6dcf2d12-2954-413d-a295-45ce38d99d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:15.745196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:15.748217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:15.748511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:15.748794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] PATH_INFO: `/v3/users/106dae758b6d41c4935e48e323b73ffb` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:15.751339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=106dae758b6d41c4935e48e323b73ffb)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:15.756768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:15.776646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 831/1646] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:15.787858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:15.788784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:15.789190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:15.822471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:15.823936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:15.833896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 832/1647] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:15.834790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:15.837139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:15.929104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff52a0-2d9c-4315-87eb-2746af358941 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:15.934498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:15.934670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:15.934916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:15.935191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:15.968418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:15.972630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:15.995382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 833/1648] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:15.999050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:16.004127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106dae758b6d41c4935e48e323b73ffb'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:16.004981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106dae758b6d41c4935e48e323b73ffb'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:16.005593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106dae758b6d41c4935e48e323b73ffb'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:16.023771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9ae81-d919-473a-9b0c-61e6640d185b service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:16.027326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:16.030330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b289dd3-7a8a-49d5-9abb-52e8d434f167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:16.030966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b289dd3-7a8a-49d5-9abb-52e8d434f167 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:16.035432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b289dd3-7a8a-49d5-9abb-52e8d434f167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:16.035432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b289dd3-7a8a-49d5-9abb-52e8d434f167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:16.037762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3b289dd3-7a8a-49d5-9abb-52e8d434f167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:16.057711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b289dd3-7a8a-49d5-9abb-52e8d434f167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:16.067076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b289dd3-7a8a-49d5-9abb-52e8d434f167 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:16.098453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106dae758b6d41c4935e48e323b73ffb'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:16.099623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 834/1649] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:16.110567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:16.112162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106dae758b6d41c4935e48e323b73ffb'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:16.163266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 816/1650] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:15 2022] DELETE /identity/v3/users/106dae758b6d41c4935e48e323b73ffb => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:16.165248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:16.165660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:16.166067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:16.209276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:16.211815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:16.314267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 835/1651] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:16.331013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:16.331993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:16.421843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783cfdb7-8386-4d7e-8a64-d88cdd85a9cc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:16.426586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:16.427517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:16.427805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:16.428079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:16.449195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:16.453491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:16.465618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 817/1652] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:17.689876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:17.691411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:17.718598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a9f33-9e88-415e-a589-bcfb9df0e855 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:17.752141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:17.753230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:17.753230 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:17.753373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] PATH_INFO: `/v3/projects/6af25d4a38fa43c496a92b6c0c66fb73` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:17.755460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af25d4a38fa43c496a92b6c0c66fb73)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:17.756439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: WARNING py.warnings [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 22 11:04:17.756439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: warnings.warn(msg) Aug 22 11:04:17.756439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: Aug 22 11:04:17.759073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:17.972110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:17.974170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:17.998946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f338e9-a51e-4a89-82cb-3b775f86ded1 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:18.040621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af25d4a38fa43c496a92b6c0c66fb73', 'request_id': 'req-18de2547-cdf3-4e8a-a864-438df037c0d0'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.044286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:18.063740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af25d4a38fa43c496a92b6c0c66fb73', 'request_id': 'req-18de2547-cdf3-4e8a-a864-438df037c0d0'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.098675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af25d4a38fa43c496a92b6c0c66fb73', 'request_id': 'req-18de2547-cdf3-4e8a-a864-438df037c0d0'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.175888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] The token cache is being invalidate because project 6af25d4a38fa43c496a92b6c0c66fb73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:18.176360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.178581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.182593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 836/1653] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:17 2022] DELETE /identity/v3/projects/6af25d4a38fa43c496a92b6c0c66fb73 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:18.186432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:18.188948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:18.213611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:18.220271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:18.222986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:18.224755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] PATH_INFO: `/v3/projects/7d88aee438994d1b96443472f132ea77` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:18.230028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d88aee438994d1b96443472f132ea77)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:18.233235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:18.235429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:18.385231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de2547-cdf3-4e8a-a864-438df037c0d0 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:18.392715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:18.393557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:18.393846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:18.394120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] PATH_INFO: `/v3/users/e3e57ecd8c5d49619298134f2b103181` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:18.395497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e57ecd8c5d49619298134f2b103181)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:18.398374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:18.680456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d88aee438994d1b96443472f132ea77', 'request_id': 'req-7e785cd0-5ee4-457f-9533-8453e79269c0'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.689565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d88aee438994d1b96443472f132ea77', 'request_id': 'req-7e785cd0-5ee4-457f-9533-8453e79269c0'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.707743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d88aee438994d1b96443472f132ea77', 'request_id': 'req-7e785cd0-5ee4-457f-9533-8453e79269c0'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.798624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] The token cache is being invalidate because project 7d88aee438994d1b96443472f132ea77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:18.800940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.802462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.807351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e57ecd8c5d49619298134f2b103181'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.809606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 818/1654] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:17 2022] DELETE /identity/v3/projects/7d88aee438994d1b96443472f132ea77 => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:18.825747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e57ecd8c5d49619298134f2b103181'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.826396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:18.830363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e57ecd8c5d49619298134f2b103181'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.832311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:18.874389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e57ecd8c5d49619298134f2b103181'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.944248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9a735be6-df80-4cc4-8858-c2f03540cdc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e57ecd8c5d49619298134f2b103181'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:18.996851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 837/1655] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:18 2022] DELETE /identity/v3/users/e3e57ecd8c5d49619298134f2b103181 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:18.996851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d4b35-19a6-4be1-a9f4-1a0271e8fa1a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:18.996851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d4b35-19a6-4be1-a9f4-1a0271e8fa1a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:18.996851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d4b35-19a6-4be1-a9f4-1a0271e8fa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.019004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-1c6d4b35-19a6-4be1-a9f4-1a0271e8fa1a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:19.020568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1c6d4b35-19a6-4be1-a9f4-1a0271e8fa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:19.065497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e785cd0-5ee4-457f-9533-8453e79269c0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:19.130727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 838/1656] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:19.139318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515a7c-e238-4188-9761-fd56e7f0119b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:19.139610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515a7c-e238-4188-9761-fd56e7f0119b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:19.139890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515a7c-e238-4188-9761-fd56e7f0119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.180714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2e515a7c-e238-4188-9761-fd56e7f0119b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:19.182305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2e515a7c-e238-4188-9761-fd56e7f0119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:19.204368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 839/1657] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:19.205341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e515a7c-e238-4188-9761-fd56e7f0119b None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:19.232531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:19.234965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:19.234965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:19.235147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e515a7c-e238-4188-9761-fd56e7f0119b None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:19.235244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] PATH_INFO: `/v3/projects/3c1502477fac4fb5aa2edcaf77cef9c2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.237572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1502477fac4fb5aa2edcaf77cef9c2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:19.239676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:19.245800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:19.246467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:19.246761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:19.247044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.247716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:19.248593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:19.251482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:19.272997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 840/1658] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:19.279398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:19.303529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fd859-bb17-41c4-9974-32bbf37df672 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:19.313993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:19.314909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:19.315360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:19.315787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.316586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:19.320356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:19.440265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1502477fac4fb5aa2edcaf77cef9c2', 'request_id': 'req-161dc952-16c5-420e-a228-8e09b7851b7a'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:19.458147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1502477fac4fb5aa2edcaf77cef9c2', 'request_id': 'req-161dc952-16c5-420e-a228-8e09b7851b7a'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:19.470637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1502477fac4fb5aa2edcaf77cef9c2', 'request_id': 'req-161dc952-16c5-420e-a228-8e09b7851b7a'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:19.509650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 841/1659] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:19 2022] POST /identity/v3/projects => generated 363 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:19.515997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:19.607490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] The token cache is being invalidate because project 3c1502477fac4fb5aa2edcaf77cef9c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:19.607966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:19.611439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:19.614267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 819/1660] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:18 2022] DELETE /identity/v3/projects/3c1502477fac4fb5aa2edcaf77cef9c2 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:19.621330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:19.629337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:19.682609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54738a20-1ec7-4870-bfa5-cf7e8d70c04f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:19.699811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:19.701343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:19.701446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:19.701879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.703871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:19.713541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:19.816489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dc952-16c5-420e-a228-8e09b7851b7a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:19.826803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:19.827712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:19.828157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:19.828575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] PATH_INFO: `/v3/users/aad8451ccc504648807b986b9ee52549` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.830872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad8451ccc504648807b986b9ee52549)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:19.834319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:19.855374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 842/1661] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:19 2022] POST /identity/v3/users => generated 282 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:04:19.862890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:19.865740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:19.904206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2cfae9-2399-4fca-8f5a-4cbdc9bc6390 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:19.919508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:19.920403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:19.920819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:19.921259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:19.935517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:19.936982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:19.976252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 843/1662] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:19.978575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:19.982521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:20.030052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6705fd-051e-4c79-8eb3-aa90c62393e0 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:20.036766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:20.037560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:20.037954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:20.038402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] PATH_INFO: `/v3/projects/a6b703bcc49f4debbfc319188b28f13b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:20.041153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b703bcc49f4debbfc319188b28f13b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:20.049278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:20.086661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad8451ccc504648807b986b9ee52549'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.090588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad8451ccc504648807b986b9ee52549'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.091986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:20.096667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad8451ccc504648807b986b9ee52549'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.178173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad8451ccc504648807b986b9ee52549'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.188056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad8451ccc504648807b986b9ee52549'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.232702 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 820/1663] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:19 2022] DELETE /identity/v3/users/aad8451ccc504648807b986b9ee52549 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:20.234414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:20.239520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:20.265922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b703bcc49f4debbfc319188b28f13b', 'request_id': 'req-c1d8fb3d-867e-45c0-9c83-8198e2b47139'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.298085 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b703bcc49f4debbfc319188b28f13b', 'request_id': 'req-c1d8fb3d-867e-45c0-9c83-8198e2b47139'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.321219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b703bcc49f4debbfc319188b28f13b', 'request_id': 'req-c1d8fb3d-867e-45c0-9c83-8198e2b47139'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.363726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:20.412850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f51b0-b880-4509-8802-c5ae8f1c72ff admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:20.425328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] The token cache is being invalidate because project a6b703bcc49f4debbfc319188b28f13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:20.425892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.427746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d8fb3d-867e-45c0-9c83-8198e2b47139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:20.449183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 844/1664] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:19 2022] DELETE /identity/v3/projects/a6b703bcc49f4debbfc319188b28f13b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:20.456675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7a3f9-e903-4be7-8d22-f6463c822043 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:20.456675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7a3f9-e903-4be7-8d22-f6463c822043 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:20.456675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7a3f9-e903-4be7-8d22-f6463c822043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:20.492417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bae7a3f9-e903-4be7-8d22-f6463c822043 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:20.493916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bae7a3f9-e903-4be7-8d22-f6463c822043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:20.561652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:20.562027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:20.562340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:20.562676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:20.563357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:20.565493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:20.578559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 821/1665] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:20.582401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:20.589145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:20.620863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4a0d5-6583-465a-bcb6-70ab846bf079 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:20.645502 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b2c60-85d6-41fd-b18d-b46c174e57dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:20.646580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b2c60-85d6-41fd-b18d-b46c174e57dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:20.647057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b2c60-85d6-41fd-b18d-b46c174e57dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:20.647505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b2c60-85d6-41fd-b18d-b46c174e57dc admin admin] PATH_INFO: `/v3/projects/e09c6dd70f8a4b8ab5dfa004257f6719/users/bcd223442a044628a146965e77752872/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:20.705614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b2c60-85d6-41fd-b18d-b46c174e57dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09c6dd70f8a4b8ab5dfa004257f6719, user_id=bcd223442a044628a146965e77752872, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:20.709373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b2c60-85d6-41fd-b18d-b46c174e57dc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:20.732402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 845/1666] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:20.736128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9731186-a705-4b96-bbe1-db9b2183d3f1 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:20.736128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9731186-a705-4b96-bbe1-db9b2183d3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:20.736128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9731186-a705-4b96-bbe1-db9b2183d3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:20.808965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 822/1667] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:20 2022] PUT /identity/v3/projects/e09c6dd70f8a4b8ab5dfa004257f6719/users/bcd223442a044628a146965e77752872/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:20.809632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d9731186-a705-4b96-bbe1-db9b2183d3f1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:20.809632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d9731186-a705-4b96-bbe1-db9b2183d3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:20.813680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991d11-1074-489e-93c7-bbdc095045db None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:20.815389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991d11-1074-489e-93c7-bbdc095045db None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:20.816167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87991d11-1074-489e-93c7-bbdc095045db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:20.914464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-87991d11-1074-489e-93c7-bbdc095045db None None] MFA Rules not processed for user `bcd223442a044628a146965e77752872`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:20.916557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-87991d11-1074-489e-93c7-bbdc095045db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:20.981970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 846/1668] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:20.986475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9731186-a705-4b96-bbe1-db9b2183d3f1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.013446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9731186-a705-4b96-bbe1-db9b2183d3f1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.020268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.021293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.021763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.022329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.023179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:21.023678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.026270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.040459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 823/1669] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:21.051674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87991d11-1074-489e-93c7-bbdc095045db None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.057733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 847/1670] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:04:21.063370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.072061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87991d11-1074-489e-93c7-bbdc095045db None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.086683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.087447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.088393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.088393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.089431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.094930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698e810-fa1f-43e5-9b9b-6cd0be5a1089 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.096724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.114640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.115606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.116109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.116584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.117425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.123355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.236255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 824/1671] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:21 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:21.241553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.296219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 848/1672] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:21 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:21.307825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.401108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c3b6-f9bf-42c4-bc15-f8c4c76d55d3 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.475206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff3ca8-911c-467c-a3b2-41c315ed3599 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.501900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.503757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.504388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.504506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.504864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.505207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.505635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.506021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.506774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.508836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.509618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.531271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.617862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 825/1673] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:21 2022] POST /identity/v3/users => generated 287 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:21.620221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.636946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74dcc6-b246-404c-bc2a-5af4296d8f19 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.645616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 849/1674] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:21 2022] POST /identity/v3/users => generated 292 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:21.649802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.653366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.654213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.658849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.659832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.661394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.664874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.668121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff0b13-5cfb-45b3-a210-1fbe992571a9 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.677241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.677878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.678147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.678478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.679654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.682043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.687063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 826/1675] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:21.688231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.701173 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 850/1676] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:04:21.702166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87938cf4-50ba-415a-a671-353a38f2ed2d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.705287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.708270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.708960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.709248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.709526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] PATH_INFO: `/v3/projects/fda2385543e142f584d04bad2b8b6ab1/users/62b4eb8c6bb04810a5ee8dfc5a8eba9f/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.726715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510384c-78a6-4528-98eb-646eb7935461 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.729258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda2385543e142f584d04bad2b8b6ab1, user_id=62b4eb8c6bb04810a5ee8dfc5a8eba9f, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.732404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.736253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.737167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.738405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.738941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.739902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.741513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.763273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 851/1677] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:21.767465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.781169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349bb21-1084-4072-8ae6-68c2af82ed28 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:21.789349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113cb053-e63e-407d-8ec2-d8e80a630e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:21.790169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb053-e63e-407d-8ec2-d8e80a630e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.791238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb053-e63e-407d-8ec2-d8e80a630e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.795114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cb053-e63e-407d-8ec2-d8e80a630e2c admin admin] PATH_INFO: `/v3/projects/b053455a4101415fb9dd6fb8c0aec4f5/users/65c8e267c373426b9c820ad544bbb43b/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:21.807626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 827/1678] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:21 2022] PUT /identity/v3/projects/fda2385543e142f584d04bad2b8b6ab1/users/62b4eb8c6bb04810a5ee8dfc5a8eba9f/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:21.812144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:21.821995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:21.873506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cb053-e63e-407d-8ec2-d8e80a630e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b053455a4101415fb9dd6fb8c0aec4f5, user_id=65c8e267c373426b9c820ad544bbb43b, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:21.881388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cb053-e63e-407d-8ec2-d8e80a630e2c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:21.968271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 852/1679] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:21 2022] PUT /identity/v3/projects/b053455a4101415fb9dd6fb8c0aec4f5/users/65c8e267c373426b9c820ad544bbb43b/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:21.975348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8fdb5-24d8-4429-8512-9f3ba341e435 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:21.975348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8fdb5-24d8-4429-8512-9f3ba341e435 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:21.975348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8fdb5-24d8-4429-8512-9f3ba341e435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.004085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543444b5-b24f-485d-9677-e59ee0242680 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:22.096849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-39a8fdb5-24d8-4429-8512-9f3ba341e435 None None] MFA Rules not processed for user `65c8e267c373426b9c820ad544bbb43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:22.098091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-39a8fdb5-24d8-4429-8512-9f3ba341e435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:22.161448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:22.195230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:22.195703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:22.199645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:22.199645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.199645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:22.203199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:22.229620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 828/1680] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:04:22.233614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:22.239918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:22.278309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8774c0-69b9-4312-a747-262704e88b3b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:22.296175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:22.303939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:22.307286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:22.307738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] PATH_INFO: `/v3/projects/9db7cc628b80407dbaf6156934a2e95c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.311734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db7cc628b80407dbaf6156934a2e95c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:22.319683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:22.323911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 853/1681] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:22.328889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8fdb5-24d8-4429-8512-9f3ba341e435 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:22.354364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8fdb5-24d8-4429-8512-9f3ba341e435 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:22.360049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:22.361060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:22.361334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:22.361613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] PATH_INFO: `/v3/projects/fda2385543e142f584d04bad2b8b6ab1/users/62b4eb8c6bb04810a5ee8dfc5a8eba9f/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.376118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda2385543e142f584d04bad2b8b6ab1, user_id=62b4eb8c6bb04810a5ee8dfc5a8eba9f, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:22.379043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:22.402765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 854/1682] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:22 2022] PUT /identity/v3/projects/fda2385543e142f584d04bad2b8b6ab1/users/62b4eb8c6bb04810a5ee8dfc5a8eba9f/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:22.403683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:22.494415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac8907-dcb4-48ae-a32f-040e2755c834 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:22.506594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650945a-9762-4129-ba41-3b800a38b04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:22.507245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650945a-9762-4129-ba41-3b800a38b04a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:22.507564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650945a-9762-4129-ba41-3b800a38b04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:22.509954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650945a-9762-4129-ba41-3b800a38b04a admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.509954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650945a-9762-4129-ba41-3b800a38b04a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:22.511384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650945a-9762-4129-ba41-3b800a38b04a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:22.558691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db7cc628b80407dbaf6156934a2e95c', 'request_id': 'req-96351468-a35f-42d2-9801-3808663e452b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:22.565756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db7cc628b80407dbaf6156934a2e95c', 'request_id': 'req-96351468-a35f-42d2-9801-3808663e452b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:22.576458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db7cc628b80407dbaf6156934a2e95c', 'request_id': 'req-96351468-a35f-42d2-9801-3808663e452b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:22.657841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 855/1683] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:22 2022] POST /identity/v3/projects => generated 385 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:22.659729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d599d5-182e-4c51-ab87-fa1185b9e153 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:22.660336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d599d5-182e-4c51-ab87-fa1185b9e153 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:22.660749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d599d5-182e-4c51-ab87-fa1185b9e153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.685430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] The token cache is being invalidate because project 9db7cc628b80407dbaf6156934a2e95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:22.685829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:22.690777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:22.693655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 829/1684] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:22 2022] DELETE /identity/v3/projects/9db7cc628b80407dbaf6156934a2e95c => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:22.694532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:22.700631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:22.706750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-71d599d5-182e-4c51-ab87-fa1185b9e153 None None] MFA Rules not processed for user `62b4eb8c6bb04810a5ee8dfc5a8eba9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:22.713275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-71d599d5-182e-4c51-ab87-fa1185b9e153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:22.844804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 856/1685] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:22.852052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96351468-a35f-42d2-9801-3808663e452b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:22.854483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:22.855380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:22.855990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.912187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:22.913384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:22.988418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdfabf-40ae-4cd6-88ad-f019bcc00540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:22.989638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfabf-40ae-4cd6-88ad-f019bcc00540 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:22.990364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfabf-40ae-4cd6-88ad-f019bcc00540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:22.990930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdfabf-40ae-4cd6-88ad-f019bcc00540 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:22.995646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdfabf-40ae-4cd6-88ad-f019bcc00540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:22.995646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdfabf-40ae-4cd6-88ad-f019bcc00540 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.053171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:23.078405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 857/1686] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:23.078405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.081080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:23.131233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 830/1687] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:22 2022] POST /identity/v3/users => generated 299 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:23.133435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6542-23e3-4fa8-b172-f17102d8d655 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.134166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6542-23e3-4fa8-b172-f17102d8d655 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.136059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6542-23e3-4fa8-b172-f17102d8d655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.153735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792281b-552f-498c-b5aa-5128f99727a4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.158577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.159319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.159668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.160622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.164113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a1dc6542-23e3-4fa8-b172-f17102d8d655 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:23.165869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a1dc6542-23e3-4fa8-b172-f17102d8d655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:23.175212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.176749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.180496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 831/1688] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:23.184943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:23.194990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 858/1689] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:23.196054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.208025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6542-23e3-4fa8-b172-f17102d8d655 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.211572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979a38e-eff7-48f2-bb5e-35a6573584d1 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.226671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.229351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.230190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.231562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6542-23e3-4fa8-b172-f17102d8d655 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.231948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.232862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:23.235627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.239314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.245459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.246109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.246410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.246684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.250116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.255952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.273921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 859/1690] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:23.276620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.291422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 832/1691] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:23.292663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.313467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a05f2-981c-49be-9d4e-ab1d77ba291f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.322165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847e030-d7f9-4f8e-ae3b-4ec4f3702fb4 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.324281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.326414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.326658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.328980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] PATH_INFO: `/v3/projects/35111c1393a94883972f8002325504aa/users/f65206003da6400bbb32e543e18761a1/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.336703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.339168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.339168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.339168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.339967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.346071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.370490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35111c1393a94883972f8002325504aa, user_id=f65206003da6400bbb32e543e18761a1, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.376144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.517049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 860/1692] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:23 2022] POST /identity/v3/projects => generated 379 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:23.518288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.518805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 833/1693] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:23 2022] PUT /identity/v3/projects/35111c1393a94883972f8002325504aa/users/f65206003da6400bbb32e543e18761a1/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:23.523210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.683493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54835e3-9b56-4923-992c-da8795902f48 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.706335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.707265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.708462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.709791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.710685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.722420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.800487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76344ce9-090a-4b21-8010-5cf381c20986 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.813461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.817692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.817692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.817692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.820762 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.824037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.849559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 861/1694] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:23 2022] POST /identity/v3/users => generated 290 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:23.858123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.877764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 834/1695] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:23.880792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.887838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a5f06-c02a-41e6-8f02-05c99e7184e3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.900581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.903626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.904037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.905670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.907258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11137160-2990-4fb3-ad00-1e3f047913d2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:23.907477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.910847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.926024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593eec5-6788-4b0a-86df-dab63b901457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:23.928259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593eec5-6788-4b0a-86df-dab63b901457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.928545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593eec5-6788-4b0a-86df-dab63b901457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.928821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593eec5-6788-4b0a-86df-dab63b901457 admin admin] PATH_INFO: `/v3/projects/35111c1393a94883972f8002325504aa/users/f65206003da6400bbb32e543e18761a1/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:23.943844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593eec5-6788-4b0a-86df-dab63b901457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35111c1393a94883972f8002325504aa, user_id=f65206003da6400bbb32e543e18761a1, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:23.953883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593eec5-6788-4b0a-86df-dab63b901457 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:23.968487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 862/1696] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:23.969887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 835/1697] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:23 2022] PUT /identity/v3/projects/35111c1393a94883972f8002325504aa/users/f65206003da6400bbb32e543e18761a1/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:23.972483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:23.982734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d5d57-2173-4f44-9493-9afd9539250c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:23.983024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d5d57-2173-4f44-9493-9afd9539250c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:23.983324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d5d57-2173-4f44-9493-9afd9539250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.033286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c9f0-0ee0-4d24-a5ce-01ad22894ae3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:24.059487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e82d5d57-2173-4f44-9493-9afd9539250c None None] MFA Rules not processed for user `f65206003da6400bbb32e543e18761a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:24.068428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e82d5d57-2173-4f44-9493-9afd9539250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:24.228372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 836/1698] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:24.230526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27a47-3701-4082-bfb7-517b2400db41 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:24.230810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27a47-3701-4082-bfb7-517b2400db41 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:24.231083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27a47-3701-4082-bfb7-517b2400db41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.278011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:24.278938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:24.279229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:24.279501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.280155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:24.281765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:24.314213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 863/1699] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:24.314213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:24.318468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6ba27a47-3701-4082-bfb7-517b2400db41 None None] MFA Rules not processed for user `7c6fee0b956e41a7b2d9b914043633ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:24.322941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6ba27a47-3701-4082-bfb7-517b2400db41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:24.332246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:24.450031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:24.450031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875a7c1-c23e-4863-8497-8db97174f407 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:24.453434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4dd72-9b95-4acb-948a-efb7e18c455f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:24.454385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4dd72-9b95-4acb-948a-efb7e18c455f service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:24.454874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4dd72-9b95-4acb-948a-efb7e18c455f service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:24.455343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4dd72-9b95-4acb-948a-efb7e18c455f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.458284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-80b4dd72-9b95-4acb-948a-efb7e18c455f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:24.471750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 837/1700] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:24.472850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba27a47-3701-4082-bfb7-517b2400db41 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:24.485429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba27a47-3701-4082-bfb7-517b2400db41 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:24.491685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:24.492610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:24.492900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:24.493179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] PATH_INFO: `/v3/projects/88906e82b9414547941f360f75972220/users/c86306b568134876a002ccf7f7f920ff/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.545535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88906e82b9414547941f360f75972220, user_id=c86306b568134876a002ccf7f7f920ff, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:24.549288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:24.614140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4dd72-9b95-4acb-948a-efb7e18c455f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:24.615813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4dd72-9b95-4acb-948a-efb7e18c455f service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:24.636961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 864/1701] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:24.638629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb7c54-bd6c-456e-90a4-01e6ca176035 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:24.639071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb7c54-bd6c-456e-90a4-01e6ca176035 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:24.639351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb7c54-bd6c-456e-90a4-01e6ca176035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.672473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-17bb7c54-bd6c-456e-90a4-01e6ca176035 None None] MFA Rules not processed for user `7c6fee0b956e41a7b2d9b914043633ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:24.675885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-17bb7c54-bd6c-456e-90a4-01e6ca176035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:24.676000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 838/1702] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:24 2022] PUT /identity/v3/projects/88906e82b9414547941f360f75972220/users/c86306b568134876a002ccf7f7f920ff/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:24.676000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:24.691681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:24.801976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 865/1703] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:24.803565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:24.808526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:24.809084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.898143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] MFA Rules not processed for user `c86306b568134876a002ccf7f7f920ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:24.901018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:24.980333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d08244-931c-4d72-96d7-cd0fa0cb229a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:24.991112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:24.995077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:24.995618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:24.996524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] PATH_INFO: `/v3/users/6e9ddaf55d0a48219dbe517c7ff4aa32` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:24.998884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e9ddaf55d0a48219dbe517c7ff4aa32)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:25.002714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:25.090223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 866/1704] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:25.091140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:25.100199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:25.298386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b715288-af6c-418f-b81e-54e885e73ef3 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:25.311919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d4d66-b06f-4697-8b19-34eeb90491f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:25.311919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d4d66-b06f-4697-8b19-34eeb90491f2 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:25.311919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d4d66-b06f-4697-8b19-34eeb90491f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:25.311919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d4d66-b06f-4697-8b19-34eeb90491f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:25.326154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d4d66-b06f-4697-8b19-34eeb90491f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:25.327854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d4d66-b06f-4697-8b19-34eeb90491f2 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:25.344373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 867/1705] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:25.347319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:25.347648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:25.347976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:25.393237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e9ddaf55d0a48219dbe517c7ff4aa32'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:25.395540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e9ddaf55d0a48219dbe517c7ff4aa32'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:25.396668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e9ddaf55d0a48219dbe517c7ff4aa32'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:25.421906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] MFA Rules not processed for user `d13b7df898df4d238ea290fab8edc4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:25.423298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:25.497640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e9ddaf55d0a48219dbe517c7ff4aa32'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:25.538274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:25.540212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e9ddaf55d0a48219dbe517c7ff4aa32'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:25.592808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 868/1706] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:25.595859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:25.604636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:25.611112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 839/1707] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:24 2022] DELETE /identity/v3/users/6e9ddaf55d0a48219dbe517c7ff4aa32 => generated 0 bytes in 935 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:25.614707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:25.629082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ec88b-6825-4695-b4a5-834080957391 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:25.631211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:25.666510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:25.850410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c433e87-2da5-404c-a8c1-b5e3b2d3b660 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:25.885254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:25.886478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:25.886478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:25.886478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] PATH_INFO: `/v3/users/bcd223442a044628a146965e77752872` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:25.901772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd223442a044628a146965e77752872)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:25.913106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:25.980055 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe50e3f-56ca-4c7a-8b45-4c5c0aa9f825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:25.981029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe50e3f-56ca-4c7a-8b45-4c5c0aa9f825 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:25.984105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe50e3f-56ca-4c7a-8b45-4c5c0aa9f825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:25.984685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe50e3f-56ca-4c7a-8b45-4c5c0aa9f825 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:25.986148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe50e3f-56ca-4c7a-8b45-4c5c0aa9f825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:25.991648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe50e3f-56ca-4c7a-8b45-4c5c0aa9f825 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:26.184984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 869/1708] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:25 2022] POST /identity/v3/projects => generated 379 bytes in 591 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:26.186743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:26.190681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:26.192695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:26.260092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:26.262867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:26.313136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd223442a044628a146965e77752872'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:26.313885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd223442a044628a146965e77752872'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:26.314525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd223442a044628a146965e77752872'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:26.406199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 870/1709] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:26.407235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:26.412142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:26.429163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd223442a044628a146965e77752872'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:26.441978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-827c2789-22b1-44c6-88a7-ee695c514461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd223442a044628a146965e77752872'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:26.495702 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 840/1710] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:25 2022] DELETE /identity/v3/users/bcd223442a044628a146965e77752872 => generated 0 bytes in 880 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:26.499333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:26.502193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:26.502976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:26.563197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] MFA Rules not processed for user `7acea30e695341c8a1d83f386629a99b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:26.564859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:26.604297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea2661-6d6c-48a3-8ae2-a78cf3da6327 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:26.700132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 841/1711] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:26.700132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:26.701425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:26.726585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06342bd8-fff7-4d38-9280-7e691da48644 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:26.730436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06342bd8-fff7-4d38-9280-7e691da48644 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:26.730436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06342bd8-fff7-4d38-9280-7e691da48644 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:26.730436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06342bd8-fff7-4d38-9280-7e691da48644 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:26.730966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-06342bd8-fff7-4d38-9280-7e691da48644 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:26.850458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e74b5-fecd-4344-8022-f110cb2c9d1b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:26.858892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf928b94-45dd-4d6e-91db-527d9a41dfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:26.865716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf928b94-45dd-4d6e-91db-527d9a41dfdd admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:26.865716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf928b94-45dd-4d6e-91db-527d9a41dfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:26.865716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf928b94-45dd-4d6e-91db-527d9a41dfdd admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:26.865716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf928b94-45dd-4d6e-91db-527d9a41dfdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:26.866139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf928b94-45dd-4d6e-91db-527d9a41dfdd admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:26.934803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06342bd8-fff7-4d38-9280-7e691da48644 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:26.937773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06342bd8-fff7-4d38-9280-7e691da48644 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:26.956126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 871/1712] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:26.958542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465c07-cfe5-4168-b42c-08bece18fc9b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:26.962691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465c07-cfe5-4168-b42c-08bece18fc9b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:26.962691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41465c07-cfe5-4168-b42c-08bece18fc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:26.976505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 842/1713] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:26 2022] POST /identity/v3/users => generated 296 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:26.976505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1cf7d-d5ef-4c62-8500-bbaffa3161fd None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:26.976505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1cf7d-d5ef-4c62-8500-bbaffa3161fd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:26.976505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1cf7d-d5ef-4c62-8500-bbaffa3161fd None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:26.983996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 843/1714] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 22 11:04:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:26.985209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1cf7d-d5ef-4c62-8500-bbaffa3161fd None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:26.988845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-41465c07-cfe5-4168-b42c-08bece18fc9b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:26.992639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b1b1cf7d-d5ef-4c62-8500-bbaffa3161fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:27.000479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-41465c07-cfe5-4168-b42c-08bece18fc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:27.010709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 872/1715] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:27.011994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32190fb9-c22f-4d3d-93d8-8eafcc864d8e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.012291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32190fb9-c22f-4d3d-93d8-8eafcc864d8e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.012725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32190fb9-c22f-4d3d-93d8-8eafcc864d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.050103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:27.074204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-32190fb9-c22f-4d3d-93d8-8eafcc864d8e None None] MFA Rules not processed for user `7acea30e695341c8a1d83f386629a99b`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:27.076846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-32190fb9-c22f-4d3d-93d8-8eafcc864d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:27.086608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 873/1716] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:27.093361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32190fb9-c22f-4d3d-93d8-8eafcc864d8e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:27.118181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32190fb9-c22f-4d3d-93d8-8eafcc864d8e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.132627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c96c22-b47d-4d2e-9ddf-71f9b523c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.133295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c96c22-b47d-4d2e-9ddf-71f9b523c87d admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.137084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c96c22-b47d-4d2e-9ddf-71f9b523c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.137355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c96c22-b47d-4d2e-9ddf-71f9b523c87d admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.138035 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c96c22-b47d-4d2e-9ddf-71f9b523c87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.146042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c96c22-b47d-4d2e-9ddf-71f9b523c87d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.156569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1cf7d-d5ef-4c62-8500-bbaffa3161fd None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.161876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.162842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.163135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.163427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.170362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 874/1717] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:27.170362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecf933-59af-46e7-9e1f-1c610c7b8bc3 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.170362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecf933-59af-46e7-9e1f-1c610c7b8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.174094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ecf933-59af-46e7-9e1f-1c610c7b8bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.193598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.198951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.221944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 844/1718] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:27.225188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:27.253194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c3367-cf0d-4330-8b7d-c7a9042b2c10 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.265684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.268705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.268998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.269269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.279752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-83ecf933-59af-46e7-9e1f-1c610c7b8bc3 None None] MFA Rules not processed for user `edd00e00b48641c8bcaacaaa7c2dd149`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:27.283344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-83ecf933-59af-46e7-9e1f-1c610c7b8bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:27.298814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.303279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 875/1719] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:27.303807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.306656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecf933-59af-46e7-9e1f-1c610c7b8bc3 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:27.326740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 845/1720] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:27.329269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ecf933-59af-46e7-9e1f-1c610c7b8bc3 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.334559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.335240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.335518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.336093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] PATH_INFO: `/v3/projects/80467a99df3e4788965e03fd8a48fb7c/users/e5fa7adc71c84397a28944bcc7528587/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.353623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80467a99df3e4788965e03fd8a48fb7c, user_id=e5fa7adc71c84397a28944bcc7528587, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.356392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.373861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:27.388015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb3f7c-8a01-4d24-9d1c-f86dd01a5ae1 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.407549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.407996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.408293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.408572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.430424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 876/1721] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:27 2022] PUT /identity/v3/projects/80467a99df3e4788965e03fd8a48fb7c/users/e5fa7adc71c84397a28944bcc7528587/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:27.433883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:27.445986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.450078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.479236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 846/1722] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:27.479967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:27.519159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d994a9-36c9-406d-a3d1-5661a8a3a453 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.567162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:27.655982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204f91-1aa5-4963-bbd8-9a26fc061197 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.687158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.687821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.688094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.688360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.688949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd218e61-22c2-406d-bfe0-00002c2252ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.689081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.691654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.692082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd218e61-22c2-406d-bfe0-00002c2252ea service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.693612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd218e61-22c2-406d-bfe0-00002c2252ea service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.696910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd218e61-22c2-406d-bfe0-00002c2252ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.721067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 847/1723] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:27.721878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:27.747364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfce121-f8dd-4680-ac2d-7ae1afeb50eb admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:27.761226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89d58a-6f00-406f-9d8f-9269dac498cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:27.761831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89d58a-6f00-406f-9d8f-9269dac498cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.762112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89d58a-6f00-406f-9d8f-9269dac498cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.762441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89d58a-6f00-406f-9d8f-9269dac498cf admin admin] PATH_INFO: `/v3/projects/80467a99df3e4788965e03fd8a48fb7c/users/e5fa7adc71c84397a28944bcc7528587/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.765600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89d58a-6f00-406f-9d8f-9269dac498cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=80467a99df3e4788965e03fd8a48fb7c, user_id=e5fa7adc71c84397a28944bcc7528587, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.770066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89d58a-6f00-406f-9d8f-9269dac498cf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.800059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 848/1724] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:27 2022] PUT /identity/v3/projects/80467a99df3e4788965e03fd8a48fb7c/users/e5fa7adc71c84397a28944bcc7528587/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:27.810956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.811559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.811973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:27.882419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] MFA Rules not processed for user `e5fa7adc71c84397a28944bcc7528587`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:27.882867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd218e61-22c2-406d-bfe0-00002c2252ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:27.884814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd218e61-22c2-406d-bfe0-00002c2252ea service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:27.888799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:27.900195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 877/1725] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:27.951309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:27.951937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:27.952628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:28.031325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] MFA Rules not processed for user `a1cdc3bfa3e046fa863131d7ff27d4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:28.033773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:28.134288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 849/1726] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:28.154880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:28.156326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:28.304791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 878/1727] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:28.368578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c66da-0085-46b9-a5d6-ffc462b04705 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:28.398034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3754d-d65d-49c7-93d0-51995e1c947f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:28.398448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3754d-d65d-49c7-93d0-51995e1c947f service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:28.398829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3754d-d65d-49c7-93d0-51995e1c947f service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:28.399205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3754d-d65d-49c7-93d0-51995e1c947f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:28.486218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3754d-d65d-49c7-93d0-51995e1c947f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:28.486218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3754d-d65d-49c7-93d0-51995e1c947f service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:28.544567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 850/1728] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:28.561352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:28.894037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:28.895776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:28.928412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:29.043078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:29.072358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164dee7-91bf-463a-8581-5eea65207929 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:29.083798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:29.086878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.088386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.088831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.098915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca3e8-f948-4d12-bf31-ac6b543c7d6f None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.100443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca3e8-f948-4d12-bf31-ac6b543c7d6f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.100443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ca3e8-f948-4d12-bf31-ac6b543c7d6f None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.101146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 851/1729] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 22 11:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:29.111264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0af28-c6f8-4459-b90a-5fb06a94d547 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.111556 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0af28-c6f8-4459-b90a-5fb06a94d547 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.111838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0af28-c6f8-4459-b90a-5fb06a94d547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.115210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:29.126114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:29.148643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 879/1730] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:29.192707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-dff0af28-c6f8-4459-b90a-5fb06a94d547 None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:29.192707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dff0af28-c6f8-4459-b90a-5fb06a94d547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:29.204217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:29.206093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:29.392027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 852/1731] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 22 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:29.393889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac66137-2374-4788-b141-724a4506650c None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.394403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac66137-2374-4788-b141-724a4506650c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.394824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac66137-2374-4788-b141-724a4506650c None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.396952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 853/1732] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 22 11:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:29.402712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac66137-2374-4788-b141-724a4506650c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:29.420333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac66137-2374-4788-b141-724a4506650c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:29.424815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517cc1b-ebcb-4ea4-9799-540286ae4e29 service nova] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:29.426800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103be4b6-27eb-4ba4-aa92-d7774254241f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:29.427896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103be4b6-27eb-4ba4-aa92-d7774254241f service placement] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.428321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103be4b6-27eb-4ba4-aa92-d7774254241f service placement] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.429013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103be4b6-27eb-4ba4-aa92-d7774254241f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.432490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-103be4b6-27eb-4ba4-aa92-d7774254241f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:29.437741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:29.438452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.442575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.442575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] PATH_INFO: `/v3/projects/b2fee426407b48138c95541c02b38637` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.442575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2fee426407b48138c95541c02b38637)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:29.442575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:29.450159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103be4b6-27eb-4ba4-aa92-d7774254241f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:29.453183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103be4b6-27eb-4ba4-aa92-d7774254241f service placement] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:29.471404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 854/1733] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:29.477751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.478340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.478827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.507174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] MFA Rules not processed for user `edd00e00b48641c8bcaacaaa7c2dd149`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:29.509747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:29.646428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2fee426407b48138c95541c02b38637', 'request_id': 'req-59771bd1-8637-477d-befe-66b83134d871'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:29.657407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 855/1734] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:29.658468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:29.660261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:29.668981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2fee426407b48138c95541c02b38637', 'request_id': 'req-59771bd1-8637-477d-befe-66b83134d871'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:29.680356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72140a9b-ebdd-4e40-aef4-970c73f08dc1 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:29.694310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2fee426407b48138c95541c02b38637', 'request_id': 'req-59771bd1-8637-477d-befe-66b83134d871'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:29.766921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:29.767939 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:29.768370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:29.768796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] PATH_INFO: `/v3/projects/e09c6dd70f8a4b8ab5dfa004257f6719` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:29.770431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09c6dd70f8a4b8ab5dfa004257f6719)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:29.773814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:29.808351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] The token cache is being invalidate because project b2fee426407b48138c95541c02b38637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:29.808793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:29.811157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:29.814377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 880/1735] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:29 2022] DELETE /identity/v3/projects/b2fee426407b48138c95541c02b38637 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:29.826271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:29.826271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:29.963420 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09c6dd70f8a4b8ab5dfa004257f6719', 'request_id': 'req-b215e5d0-a11f-4245-ab75-ed82bd046cef'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:29.966580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59771bd1-8637-477d-befe-66b83134d871 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:29.996840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09c6dd70f8a4b8ab5dfa004257f6719', 'request_id': 'req-b215e5d0-a11f-4245-ab75-ed82bd046cef'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.012609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09c6dd70f8a4b8ab5dfa004257f6719', 'request_id': 'req-b215e5d0-a11f-4245-ab75-ed82bd046cef'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.075997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] The token cache is being invalidate because project e09c6dd70f8a4b8ab5dfa004257f6719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:30.076192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.077112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b215e5d0-a11f-4245-ab75-ed82bd046cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.080139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 856/1736] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:29 2022] DELETE /identity/v3/projects/e09c6dd70f8a4b8ab5dfa004257f6719 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:30.083215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:30.083215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:30.083215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:30.099997 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be34ef-d6a5-4f15-abb5-2c8dd30f7b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:30.101023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be34ef-d6a5-4f15-abb5-2c8dd30f7b1e service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:30.101881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be34ef-d6a5-4f15-abb5-2c8dd30f7b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:30.102396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be34ef-d6a5-4f15-abb5-2c8dd30f7b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:30.106111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-92be34ef-d6a5-4f15-abb5-2c8dd30f7b1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:30.119466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be34ef-d6a5-4f15-abb5-2c8dd30f7b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:30.121465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be34ef-d6a5-4f15-abb5-2c8dd30f7b1e service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:30.134371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:30.135673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:30.159687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 881/1737] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:30.163001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:30.164597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:30.166313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:30.209034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] MFA Rules not processed for user `7c6fee0b956e41a7b2d9b914043633ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:30.211742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:30.306453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 882/1738] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:30.308844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:30.310487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:30.322883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 857/1739] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:30.322883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:30.336607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:30.346372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9ff9-d356-4a55-8fd4-fed4a7a37890 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:30.354218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:30.355090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:30.355495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:30.355972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] PATH_INFO: `/v3/users/62b4eb8c6bb04810a5ee8dfc5a8eba9f` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:30.357535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b4eb8c6bb04810a5ee8dfc5a8eba9f)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:30.361061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:30.516138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c84164-e62b-413a-981b-fce4d9fab487 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:30.532915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5745f37-a301-43a5-a217-f441145e1dc8 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5fa7adc71c84397a28944bcc7528587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80467a99df3e4788965e03fd8a48fb7c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:30.533561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5745f37-a301-43a5-a217-f441145e1dc8 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:30.533845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5745f37-a301-43a5-a217-f441145e1dc8 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:30.534117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5745f37-a301-43a5-a217-f441145e1dc8 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:30.534815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5745f37-a301-43a5-a217-f441145e1dc8 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:30.540853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5745f37-a301-43a5-a217-f441145e1dc8 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:30.540853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5745f37-a301-43a5-a217-f441145e1dc8 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:30.570609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 858/1740] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:30.574488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32335a-970e-4723-9281-b60f88c607ae None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:30.574958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32335a-970e-4723-9281-b60f88c607ae None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:30.575385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32335a-970e-4723-9281-b60f88c607ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:30.601788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-cd32335a-970e-4723-9281-b60f88c607ae None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:30.604498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cd32335a-970e-4723-9281-b60f88c607ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:30.621681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 859/1741] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:30.622898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32335a-970e-4723-9281-b60f88c607ae None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:30.642952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:30.649903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32335a-970e-4723-9281-b60f88c607ae None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:30.667851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5fa7adc71c84397a28944bcc7528587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80467a99df3e4788965e03fd8a48fb7c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:30.669081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:30.669507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:30.670261 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:30.671400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:30.673581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:30.676717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b4eb8c6bb04810a5ee8dfc5a8eba9f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.678373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b4eb8c6bb04810a5ee8dfc5a8eba9f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.682348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b4eb8c6bb04810a5ee8dfc5a8eba9f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.712547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b4eb8c6bb04810a5ee8dfc5a8eba9f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.768732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 860/1742] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:30 2022] POST /identity/v3/projects => generated 353 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:30.769949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:30.773708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:30.795515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b4eb8c6bb04810a5ee8dfc5a8eba9f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:30.846439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 883/1743] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:30 2022] DELETE /identity/v3/users/62b4eb8c6bb04810a5ee8dfc5a8eba9f => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:30.849699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:30.856357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:31.002773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeb8e1-1ea2-48e9-83a5-cb43623e0a0f tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.004914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732701b5-84b9-4562-b64d-b2d03cc1721e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.012828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5fa7adc71c84397a28944bcc7528587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80467a99df3e4788965e03fd8a48fb7c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.015483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.015971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.016715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.018060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:31.018727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.020417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:31.049184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 884/1744] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:31.065530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:31.108345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:31.123114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.123756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.124092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.124371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.124616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e766d9-3ecc-4434-995b-a1c3752ab790 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.125048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:31.125477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.127390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:31.138822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5fa7adc71c84397a28944bcc7528587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80467a99df3e4788965e03fd8a48fb7c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.139490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.139791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.140061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] PATH_INFO: `/v3/projects/14d4357bbeb64770aa154a65f0308e78` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.141307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorizing `identity:delete_project(project_id=14d4357bbeb64770aa154a65f0308e78)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.158457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:31.176751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 861/1745] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:31.182253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:31.201280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4818b7-f6cc-470e-9f56-99f008ff5cb6 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.208752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.209596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.209879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.210178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.210862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.215116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:31.411703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d4357bbeb64770aa154a65f0308e78', 'request_id': 'req-a4c6a26f-1a50-4128-9539-227e9948f049'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:31.420933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 862/1746] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:31 2022] POST /identity/v3/projects => generated 365 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:31.422777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d4357bbeb64770aa154a65f0308e78', 'request_id': 'req-a4c6a26f-1a50-4128-9539-227e9948f049'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:31.425227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:31.454020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d4357bbeb64770aa154a65f0308e78', 'request_id': 'req-a4c6a26f-1a50-4128-9539-227e9948f049'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:31.555733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] The token cache is being invalidate because project 14d4357bbeb64770aa154a65f0308e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:31.556069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:31.557037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:31.559463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 885/1747] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:31 2022] DELETE /identity/v3/projects/14d4357bbeb64770aa154a65f0308e78 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:31.640529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5da5b-5bcb-42e6-bdfb-d92e8033aabe admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.650048 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.652112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.652782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.653220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.654094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.657931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:31.711884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:31.715502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:31.748646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 863/1748] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:31 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:31.753860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:31.756850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:31.796422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d01c-311e-4367-b5f5-06934d7829ec admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.806153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.807468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.808968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.809444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.810527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.813947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:31.829137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 864/1749] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:31.833924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:31.875355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb17af3-fc69-45d8-bde0-e42eedf1f597 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.882299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.883191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.883706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.884136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.885235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.887139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:31.904277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 865/1750] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:04:31.911787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:31.940264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac503b-595a-454b-aa11-e30967dd85fa admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.954695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78635e3-5c6b-4ad1-aab5-33b4ac817c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.958477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78635e3-5c6b-4ad1-aab5-33b4ac817c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.959241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78635e3-5c6b-4ad1-aab5-33b4ac817c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.960253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78635e3-5c6b-4ad1-aab5-33b4ac817c1c admin admin] PATH_INFO: `/v3/projects/be3ee0b1d7e347c6b9e3ab3dd6c75428/users/7aba792e2ea646fb8006e74e524952e6/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.961372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6a26f-1a50-4128-9539-227e9948f049 tempest-VolumeQuotaClassesTest-191071445 tempest-VolumeQuotaClassesTest-191071445-project-admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:31.975937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:31.976606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:31.979581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:31.979966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] PATH_INFO: `/v3/projects/fda2385543e142f584d04bad2b8b6ab1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:31.982762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda2385543e142f584d04bad2b8b6ab1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:31.992204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:32.001673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78635e3-5c6b-4ad1-aab5-33b4ac817c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3ee0b1d7e347c6b9e3ab3dd6c75428, user_id=7aba792e2ea646fb8006e74e524952e6, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:32.013137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78635e3-5c6b-4ad1-aab5-33b4ac817c1c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:32.091955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:32.110124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 866/1751] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:31 2022] PUT /identity/v3/projects/be3ee0b1d7e347c6b9e3ab3dd6c75428/users/7aba792e2ea646fb8006e74e524952e6/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:32.118787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:32.119381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:32.119997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:32.205966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda2385543e142f584d04bad2b8b6ab1', 'request_id': 'req-d13687d9-69ef-4b64-83f4-9b9383f946ab'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:32.218946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] MFA Rules not processed for user `7aba792e2ea646fb8006e74e524952e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:32.219433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda2385543e142f584d04bad2b8b6ab1', 'request_id': 'req-d13687d9-69ef-4b64-83f4-9b9383f946ab'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:32.220693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:32.237055 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda2385543e142f584d04bad2b8b6ab1', 'request_id': 'req-d13687d9-69ef-4b64-83f4-9b9383f946ab'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:32.356337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 867/1752] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:32.360455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:32.363160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:32.384687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] The token cache is being invalidate because project fda2385543e142f584d04bad2b8b6ab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:32.384923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:32.386695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:32.389313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 886/1753] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:31 2022] DELETE /identity/v3/projects/fda2385543e142f584d04bad2b8b6ab1 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:32.390438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:32.391782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:32.525602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702c925-937a-4ec1-851a-15969eb3ea81 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:32.546834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13687d9-69ef-4b64-83f4-9b9383f946ab admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:32.552441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:32.552441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:32.552441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:32.552441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:32.556038 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:32.568782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469957a0-c465-4976-ad03-0253984bf6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:32.570224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469957a0-c465-4976-ad03-0253984bf6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:32.570693 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469957a0-c465-4976-ad03-0253984bf6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:32.571185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469957a0-c465-4976-ad03-0253984bf6a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:32.573142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469957a0-c465-4976-ad03-0253984bf6a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:32.576236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469957a0-c465-4976-ad03-0253984bf6a8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:32.697887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:32.705256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:32.714982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 887/1754] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:32 2022] POST /identity/v3/projects => generated 369 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:32.714982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c0487-d37d-4419-b2cf-a1a8e1dc381b None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:32.714982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c0487-d37d-4419-b2cf-a1a8e1dc381b None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:32.714982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c0487-d37d-4419-b2cf-a1a8e1dc381b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:32.740367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 868/1755] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:32.742677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:32.748013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:32.761370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f31c0487-d37d-4419-b2cf-a1a8e1dc381b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:32.762664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f31c0487-d37d-4419-b2cf-a1a8e1dc381b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:32.796181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259e7a7-4aba-4804-b482-7bf4db8e6933 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:32.914145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 888/1756] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:32.924019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c0487-d37d-4419-b2cf-a1a8e1dc381b None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:32.950023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:32.950023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:32.951782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:32.952203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] PATH_INFO: `/v3/users/c86306b568134876a002ccf7f7f920ff` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:32.968162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c0487-d37d-4419-b2cf-a1a8e1dc381b None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:32.984926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd37ae-3888-43f4-8dee-ef74034bfd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:32.985164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86306b568134876a002ccf7f7f920ff)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:32.986509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd37ae-3888-43f4-8dee-ef74034bfd03 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:32.987380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd37ae-3888-43f4-8dee-ef74034bfd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:32.991352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd37ae-3888-43f4-8dee-ef74034bfd03 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:32.992304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd37ae-3888-43f4-8dee-ef74034bfd03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:32.996772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:32.997858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd37ae-3888-43f4-8dee-ef74034bfd03 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:33.029209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:33.100197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 889/1757] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:32 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:33.106956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2019e8-d42d-4c2d-840b-cb7b78790d45 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.107344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2019e8-d42d-4c2d-840b-cb7b78790d45 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.107696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2019e8-d42d-4c2d-840b-cb7b78790d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.189899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9c2019e8-d42d-4c2d-840b-cb7b78790d45 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:33.193682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9c2019e8-d42d-4c2d-840b-cb7b78790d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:33.233196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 890/1758] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:33.235998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2019e8-d42d-4c2d-840b-cb7b78790d45 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:33.276854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2019e8-d42d-4c2d-840b-cb7b78790d45 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:33.289037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:33.290586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.290586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.290990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.291805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:33.295639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:33.318180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 891/1759] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:33.323323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:33.338064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86306b568134876a002ccf7f7f920ff'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:33.340201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86306b568134876a002ccf7f7f920ff'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:33.347140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86306b568134876a002ccf7f7f920ff'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:33.356161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882d341-2ad1-41e5-8c23-80d9e46fe17e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:33.367571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:33.375702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:33.375702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.375702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.376187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.377164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:33.377570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:33.388306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:33.417240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 892/1760] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:33.421918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:33.421918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:33.448996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c6419-05b1-46df-aeed-13c4442478d4 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:33.449217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86306b568134876a002ccf7f7f920ff'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:33.449318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051ee2e-b350-41b2-8a7f-d7c634d9bf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:33.449318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ee2e-b350-41b2-8a7f-d7c634d9bf01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.449318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ee2e-b350-41b2-8a7f-d7c634d9bf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.449318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051ee2e-b350-41b2-8a7f-d7c634d9bf01 admin admin] PATH_INFO: `/v3/projects/388ddb2ce7a74666afaba1b140a4b081/users/6209170192a34ea2b9483cf784022d99/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.462778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86306b568134876a002ccf7f7f920ff'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:33.469886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ee2e-b350-41b2-8a7f-d7c634d9bf01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=388ddb2ce7a74666afaba1b140a4b081, user_id=6209170192a34ea2b9483cf784022d99, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:33.472842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051ee2e-b350-41b2-8a7f-d7c634d9bf01 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:33.516686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 869/1761] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:32 2022] DELETE /identity/v3/users/c86306b568134876a002ccf7f7f920ff => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:33.516686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:33.518018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:33.619867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 893/1762] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:33 2022] PUT /identity/v3/projects/388ddb2ce7a74666afaba1b140a4b081/users/6209170192a34ea2b9483cf784022d99/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:33.623713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd689d1-b016-494f-8aa3-04c9eb097911 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.624016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd689d1-b016-494f-8aa3-04c9eb097911 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.625037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd689d1-b016-494f-8aa3-04c9eb097911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.692757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-efd689d1-b016-494f-8aa3-04c9eb097911 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:33.694628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-efd689d1-b016-494f-8aa3-04c9eb097911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:33.749986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f86170-49c8-406a-a0a6-3056a1c81d1f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:33.894418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 894/1763] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:33.895723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd689d1-b016-494f-8aa3-04c9eb097911 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:33.910196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd689d1-b016-494f-8aa3-04c9eb097911 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:33.940247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d415d-fc27-4b30-8de9-d5ce85af0ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:33.942748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d415d-fc27-4b30-8de9-d5ce85af0ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.943263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d415d-fc27-4b30-8de9-d5ce85af0ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.944965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d415d-fc27-4b30-8de9-d5ce85af0ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.948111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d415d-fc27-4b30-8de9-d5ce85af0ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:33.951652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d415d-fc27-4b30-8de9-d5ce85af0ca9 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:33.981272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 895/1764] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:33.988603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd575ea-d2a5-40e3-a856-fdac78671153 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.988893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd575ea-d2a5-40e3-a856-fdac78671153 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.989172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd575ea-d2a5-40e3-a856-fdac78671153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.998430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:33.998430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:33.998430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:33.998430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:33.998849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:33.998849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:34.029768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-5bd575ea-d2a5-40e3-a856-fdac78671153 None None] MFA Rules not processed for user `d13b7df898df4d238ea290fab8edc4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:34.041584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5bd575ea-d2a5-40e3-a856-fdac78671153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:34.156181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 870/1765] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:33 2022] POST /identity/v3/projects => generated 365 bytes in 639 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:34.156181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:34.164346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:34.290729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 896/1766] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:34.298803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd575ea-d2a5-40e3-a856-fdac78671153 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:34.336834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd575ea-d2a5-40e3-a856-fdac78671153 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:34.373622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740f039-ec5a-4b44-a132-240f41cda805 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:34.392639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:34.394811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:34.395629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:34.395741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:34.643675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:34.731462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:34.739124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:34.751465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2bb0c-0b15-4a1c-bc2f-ee15b0ce716d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:34.757429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2bb0c-0b15-4a1c-bc2f-ee15b0ce716d admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:34.757429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2bb0c-0b15-4a1c-bc2f-ee15b0ce716d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:34.757429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2bb0c-0b15-4a1c-bc2f-ee15b0ce716d admin admin] PATH_INFO: `/v3/projects/388ddb2ce7a74666afaba1b140a4b081/users/6209170192a34ea2b9483cf784022d99/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:34.768272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 871/1767] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:34.769175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:34.772157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2bb0c-0b15-4a1c-bc2f-ee15b0ce716d admin admin] RBAC: Authorizing `identity:create_grant(project_id=388ddb2ce7a74666afaba1b140a4b081, user_id=6209170192a34ea2b9483cf784022d99, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:34.775818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2bb0c-0b15-4a1c-bc2f-ee15b0ce716d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:34.798746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02547982-d73b-4b85-82fd-ade2089598be service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:34.804490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:34.832259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:34.836011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:34.852275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:34.852660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:34.859030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 897/1768] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:34 2022] PUT /identity/v3/projects/388ddb2ce7a74666afaba1b140a4b081/users/6209170192a34ea2b9483cf784022d99/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:34.859836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:34.865469 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:34.872936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6fd7-aee6-4ca1-b3da-ffe9b63f76ea None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:34.879638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6fd7-aee6-4ca1-b3da-ffe9b63f76ea None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:34.879638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6fd7-aee6-4ca1-b3da-ffe9b63f76ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:34.956734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f20c6fd7-aee6-4ca1-b3da-ffe9b63f76ea None None] MFA Rules not processed for user `6209170192a34ea2b9483cf784022d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:34.958073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f20c6fd7-aee6-4ca1-b3da-ffe9b63f76ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:35.028002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 872/1769] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:34 2022] POST /identity/v3/users => generated 283 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:35.031830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:35.065062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bab467-823c-4c65-be2b-a170df3839e2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:35.224931 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 898/1770] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:35.226331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5369477-2784-4c72-81a8-b93878555c9a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:35.226620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5369477-2784-4c72-81a8-b93878555c9a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:35.227898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5369477-2784-4c72-81a8-b93878555c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:35.238194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:35.239296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:35.239741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:35.240365 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:35.241240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:35.243272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:35.258870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 873/1771] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:04:35.260137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:35.264364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:35.368503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d5369477-2784-4c72-81a8-b93878555c9a None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:35.369367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d5369477-2784-4c72-81a8-b93878555c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:35.424559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd5126-b313-449f-a00f-9cc2112070aa admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:35.445149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:35.446491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:35.446835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:35.447232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:35.488435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:35.493294 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:35.517648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 899/1772] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:35.518786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5369477-2784-4c72-81a8-b93878555c9a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:35.526149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 874/1773] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:35.551038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5369477-2784-4c72-81a8-b93878555c9a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:35.557005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:35.557839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:35.558292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:35.558745 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:35.559534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:35.561288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:35.583061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 900/1774] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:35.588157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:35.606711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671949a1-dd16-4a76-b16b-d6ef5ce09585 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:35.612717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d83bd7-95aa-4c65-98e1-dff832af2348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:35.614064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d83bd7-95aa-4c65-98e1-dff832af2348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:35.614605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d83bd7-95aa-4c65-98e1-dff832af2348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:35.615180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d83bd7-95aa-4c65-98e1-dff832af2348 admin admin] PATH_INFO: `/v3/projects/1fec90bd753a4dd2a61314818fda6255/users/b1fcb766ef9044c4badf65337a6cc7a7/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:35.638070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d83bd7-95aa-4c65-98e1-dff832af2348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fec90bd753a4dd2a61314818fda6255, user_id=b1fcb766ef9044c4badf65337a6cc7a7, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:35.641290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d83bd7-95aa-4c65-98e1-dff832af2348 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:35.699390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:35.726444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:35.742334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 875/1775] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:35 2022] PUT /identity/v3/projects/1fec90bd753a4dd2a61314818fda6255/users/b1fcb766ef9044c4badf65337a6cc7a7/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:35.753802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c645a8-dad3-4bf5-b209-c2a1fd8caa1e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:35.757983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:35.759016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:35.759462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:35.866516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] MFA Rules not processed for user `b1fcb766ef9044c4badf65337a6cc7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:35.882425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:35.991068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:35.993472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:35.993755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:35.996162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:36.003255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:36.205327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 876/1776] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:36.207004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:36.208745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:36.282942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:36.287139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:36.313778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 901/1777] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:36.314845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:36.412372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafea51-e842-435e-9816-53881fe2b5e9 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:36.420129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:36.434065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:36.434657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:36.437361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:36.438193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:36.439088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:36.442508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:36.485200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd5459-fe5d-4ecd-8212-a2e947025e1b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:36.491792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794369cd-0802-4155-8510-4726440542eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:36.495669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794369cd-0802-4155-8510-4726440542eb service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:36.495669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794369cd-0802-4155-8510-4726440542eb service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:36.495669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794369cd-0802-4155-8510-4726440542eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:36.589252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 902/1778] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:36 2022] POST /identity/v3/projects => generated 365 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:36.593330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:36.660840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794369cd-0802-4155-8510-4726440542eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:36.667950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794369cd-0802-4155-8510-4726440542eb service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:36.711452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 877/1779] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:04:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:04:36.882280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f7631-3384-441e-b8e9-0d65f44226b1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:36.901122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:36.902304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:36.902304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:36.902581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:36.903436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:36.905964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:37.064707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 903/1780] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:36 2022] POST /identity/v3/users => generated 289 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:37.066051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794369cd-0802-4155-8510-4726440542eb service swift] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:37.092842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794369cd-0802-4155-8510-4726440542eb service swift] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:37.122919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:37.123923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:37.124413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:37.125037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:37.127220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:37.130500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:37.148130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 878/1781] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:37.150360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:37.166295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066a10e-940e-4818-bc4f-ce3a5804d912 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:37.184117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:37.186692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:37.186692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:37.186692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] PATH_INFO: `/v3/projects/a0ebde3397b04716a49043896ff32827/users/5ff3d22c3e194dc8987770e5a55c41ae/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:37.225082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ebde3397b04716a49043896ff32827, user_id=5ff3d22c3e194dc8987770e5a55c41ae, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:37.235188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:37.326741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:37.327662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 904/1782] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:37 2022] PUT /identity/v3/projects/a0ebde3397b04716a49043896ff32827/users/5ff3d22c3e194dc8987770e5a55c41ae/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:37.352769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb18b9-36c8-4144-9826-f14ee33259ac admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:37.510312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:37.596681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0025d3d-7bb0-4850-88d1-4801873b40e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:37.601485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0025d3d-7bb0-4850-88d1-4801873b40e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:37.601826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0025d3d-7bb0-4850-88d1-4801873b40e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:37.602355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0025d3d-7bb0-4850-88d1-4801873b40e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:37.604324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0025d3d-7bb0-4850-88d1-4801873b40e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:37.614726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0025d3d-7bb0-4850-88d1-4801873b40e5 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:37.688447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:37.691730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 879/1783] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:37.726172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dce200-adbf-4b49-b2ff-f3ed60bfdd9a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:37.739189 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:37.741479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:37.743059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:37.746379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] PATH_INFO: `/v3/projects/a0ebde3397b04716a49043896ff32827/users/5ff3d22c3e194dc8987770e5a55c41ae/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:37.751227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ebde3397b04716a49043896ff32827, user_id=5ff3d22c3e194dc8987770e5a55c41ae, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:37.760978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:37.785653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 905/1784] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:37 2022] PUT /identity/v3/projects/a0ebde3397b04716a49043896ff32827/users/5ff3d22c3e194dc8987770e5a55c41ae/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:37.799494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0a718-91ac-4a58-8451-e52f8bd7117c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:37.799803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0a718-91ac-4a58-8451-e52f8bd7117c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:37.800093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0a718-91ac-4a58-8451-e52f8bd7117c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:37.917575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ebe0a718-91ac-4a58-8451-e52f8bd7117c None None] MFA Rules not processed for user `5ff3d22c3e194dc8987770e5a55c41ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:37.919789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ebe0a718-91ac-4a58-8451-e52f8bd7117c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:38.166434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:38.166434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:38.203301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 880/1785] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:38.239350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:38.244422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:38.247007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:38.323430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] MFA Rules not processed for user `a1cdc3bfa3e046fa863131d7ff27d4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:38.326798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:38.363557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:38.414002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a43608-389d-4b89-bdb3-d15129cbef24 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:38.427764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:38.428799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:38.429319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:38.430732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] PATH_INFO: `/v3/projects/88906e82b9414547941f360f75972220` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:38.441235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=88906e82b9414547941f360f75972220)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:38.447627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:38.590679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 881/1786] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:38.596696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:38.603149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:38.726151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88906e82b9414547941f360f75972220', 'request_id': 'req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:38.754783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88906e82b9414547941f360f75972220', 'request_id': 'req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:38.771121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88906e82b9414547941f360f75972220', 'request_id': 'req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:38.851633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff1e2b-dcb4-444b-99a3-f1720fa3bf20 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:38.921744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:38.922701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] The token cache is being invalidate because project 88906e82b9414547941f360f75972220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:38.923106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:38.924608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e7bd13d7-0151-4d7a-a4a7-9ff3e1ebd6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:38.931214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 906/1787] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:38 2022] DELETE /identity/v3/projects/88906e82b9414547941f360f75972220 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:38.931214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:38.931214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:38.931214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:39.011804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] MFA Rules not processed for user `edd00e00b48641c8bcaacaaa7c2dd149`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:39.026175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:39.043407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:39.044054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:39.044336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:39.044710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:39.047864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:39.295894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 907/1788] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Aug 22 11:04:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:39.298533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:39.298533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:39.318732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:39.320212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:39.356272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 882/1789] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:39.402117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:39.404719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:39.433046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef29fa-9a1d-4384-89d1-686a485ea234 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:39.451826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:39.451826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:39.451826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:39.452115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:39.469989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:39.471674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:39.493209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 883/1790] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:04:39.520751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ddca8-df3c-4cdb-a269-4f8847ed0938 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:39.538048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:39.538720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:39.539014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:39.543691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] PATH_INFO: `/v3/users/e5fa7adc71c84397a28944bcc7528587` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:39.543691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5fa7adc71c84397a28944bcc7528587)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:39.558729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:39.634211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:39.716262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:39.847480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5fa7adc71c84397a28944bcc7528587'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:39.851200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5fa7adc71c84397a28944bcc7528587'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:39.860860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5fa7adc71c84397a28944bcc7528587'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:39.886419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5fa7adc71c84397a28944bcc7528587'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:39.972369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5fa7adc71c84397a28944bcc7528587'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:40.025799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 908/1791] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:39 2022] DELETE /identity/v3/users/e5fa7adc71c84397a28944bcc7528587 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:40.151069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:40.152864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:40.199811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667ce47-e9b8-4439-9efd-d71294bdf8fc service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:40.348680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3d7cb-e54b-4f18-87c6-638f091e4ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:40.350099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3d7cb-e54b-4f18-87c6-638f091e4ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:40.351000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3d7cb-e54b-4f18-87c6-638f091e4ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:40.351864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3d7cb-e54b-4f18-87c6-638f091e4ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:40.356830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-82f3d7cb-e54b-4f18-87c6-638f091e4ed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:40.590180 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3d7cb-e54b-4f18-87c6-638f091e4ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:40.593111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f3d7cb-e54b-4f18-87c6-638f091e4ed1 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:40.620052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 884/1792] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:40.688410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:40.840430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:41.263349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:41.267919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:41.465401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c2a8e-7b3c-40e0-ae48-a15cd1e2c9ba admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:41.490969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:41.492059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:41.492700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:41.493186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] PATH_INFO: `/v3/users/65c8e267c373426b9c820ad544bbb43b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:41.502532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c8e267c373426b9c820ad544bbb43b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:41.506079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:41.704103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:41.870381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c8e267c373426b9c820ad544bbb43b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:41.872942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c8e267c373426b9c820ad544bbb43b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:41.874972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c8e267c373426b9c820ad544bbb43b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:41.938438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c8e267c373426b9c820ad544bbb43b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:42.063713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c8e267c373426b9c820ad544bbb43b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:42.153052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 909/1793] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:41 2022] DELETE /identity/v3/users/65c8e267c373426b9c820ad544bbb43b => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:42.158804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:42.162745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:42.162745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:42.173430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:42.247140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:42.251208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:42.355646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:42.436613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:42.438828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:42.446978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 885/1794] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:42.495296 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1eb9d-28cc-4bba-9635-d8cefd74a9f3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:42.506716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:42.507611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:42.508674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:42.509131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] PATH_INFO: `/v3/projects/80467a99df3e4788965e03fd8a48fb7c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:42.512793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=80467a99df3e4788965e03fd8a48fb7c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:42.515083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:42.529797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:42.531802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:42.669057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80467a99df3e4788965e03fd8a48fb7c', 'request_id': 'req-196e3f2b-1b51-4081-9602-c1bb7e5480bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:42.681418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d15c1b-0687-4182-b552-b7b3df0e547e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:42.683208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80467a99df3e4788965e03fd8a48fb7c', 'request_id': 'req-196e3f2b-1b51-4081-9602-c1bb7e5480bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:42.695747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80467a99df3e4788965e03fd8a48fb7c', 'request_id': 'req-196e3f2b-1b51-4081-9602-c1bb7e5480bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:42.792356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e04a2-b663-4955-a5d7-b9564a04d7eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:42.793328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e04a2-b663-4955-a5d7-b9564a04d7eb service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:42.793887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e04a2-b663-4955-a5d7-b9564a04d7eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:42.794445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e04a2-b663-4955-a5d7-b9564a04d7eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:42.817441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] The token cache is being invalidate because project 80467a99df3e4788965e03fd8a48fb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:42.817441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:42.817441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-196e3f2b-1b51-4081-9602-c1bb7e5480bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:42.818457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 910/1795] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:42 2022] DELETE /identity/v3/projects/80467a99df3e4788965e03fd8a48fb7c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:42.825478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c251863-83d9-4793-88c0-63453305fc24 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:42.826429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c251863-83d9-4793-88c0-63453305fc24 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:42.826710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c251863-83d9-4793-88c0-63453305fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:42.830026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:42.914469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9c251863-83d9-4793-88c0-63453305fc24 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:42.915781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9c251863-83d9-4793-88c0-63453305fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:43.025752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e04a2-b663-4955-a5d7-b9564a04d7eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:43.028326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e04a2-b663-4955-a5d7-b9564a04d7eb service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:43.058206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 886/1796] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:04:43.169837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 911/1797] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:43.173310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639130b3-014f-4bff-80a7-b69f8423d592 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:43.174356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639130b3-014f-4bff-80a7-b69f8423d592 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:43.175193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639130b3-014f-4bff-80a7-b69f8423d592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:43.254751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-639130b3-014f-4bff-80a7-b69f8423d592 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:43.256576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-639130b3-014f-4bff-80a7-b69f8423d592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:43.296966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 887/1798] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:43.302662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c251863-83d9-4793-88c0-63453305fc24 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:43.308601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:43.316499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c251863-83d9-4793-88c0-63453305fc24 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:43.332247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:43.332884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:43.333170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:43.333497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:43.334172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:43.334502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:43.336713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:43.366931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639130b3-014f-4bff-80a7-b69f8423d592 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:43.377867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 912/1799] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:43.380738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639130b3-014f-4bff-80a7-b69f8423d592 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:43.394889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:43.395660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:43.395884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:43.396282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:43.396801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:43.399927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:43.409373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:43.539550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:43.545216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 888/1800] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:43 2022] POST /identity/v3/projects => generated 409 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:43.561875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2eaa16-db1f-4203-a2d3-a35a7aa150e4 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:43.709848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:43.711334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:43.711780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:43.712198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:43.713147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:43.716648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:43.846597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:43.864348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 913/1801] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:43 2022] POST /identity/v3/users => generated 305 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:43.869026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:43.893545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f323e-12b0-4b70-a224-b5a15110b6b9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:43.900965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:43.902035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:43.902586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:43.908041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:43.908041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:43.908041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:43.964993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 889/1802] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:43.979815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:44.005146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bc48f-3bf4-4ea5-814f-74b56b73420b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:44.030634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a225df-2ed3-4c66-a029-a6c1ab37f428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:44.031480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a225df-2ed3-4c66-a029-a6c1ab37f428 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:44.031962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a225df-2ed3-4c66-a029-a6c1ab37f428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:44.032407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a225df-2ed3-4c66-a029-a6c1ab37f428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:44.033426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a225df-2ed3-4c66-a029-a6c1ab37f428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:44.035277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a225df-2ed3-4c66-a029-a6c1ab37f428 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:44.059843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 914/1803] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:44.064453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:44.080909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddead5-0bf2-46bd-8213-1e311f64d4e9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:44.091519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:44.092269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:44.092904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:44.096738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] PATH_INFO: `/v3/projects/5674a257263b4fa7822291d25ca2ebeb/users/e216685fa53143a9b90563a0a44908bb/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:44.154147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5674a257263b4fa7822291d25ca2ebeb, user_id=e216685fa53143a9b90563a0a44908bb, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:44.157711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:44.281269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 890/1804] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:44 2022] PUT /identity/v3/projects/5674a257263b4fa7822291d25ca2ebeb/users/e216685fa53143a9b90563a0a44908bb/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:44.283393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53318-bd6e-4204-87fb-32d6f8e67630 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:44.283739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53318-bd6e-4204-87fb-32d6f8e67630 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:44.284089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b53318-bd6e-4204-87fb-32d6f8e67630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:44.320640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:44.325643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:44.376013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-65b53318-bd6e-4204-87fb-32d6f8e67630 None None] MFA Rules not processed for user `e216685fa53143a9b90563a0a44908bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:44.377201 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-65b53318-bd6e-4204-87fb-32d6f8e67630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:44.408139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:44.455766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e846e-933e-4084-9d11-debb4c9027f8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:44.468993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:44.470776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:44.472159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:44.472590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] PATH_INFO: `/v3/users/7aba792e2ea646fb8006e74e524952e6` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:44.477396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aba792e2ea646fb8006e74e524952e6)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:44.483394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:44.588180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 915/1805] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:44.589612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53318-bd6e-4204-87fb-32d6f8e67630 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:44.618739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b53318-bd6e-4204-87fb-32d6f8e67630 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:44.629320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:44.631444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:44.635909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:44.635909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:44.635909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:44.635909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:44.641006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:44.751219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aba792e2ea646fb8006e74e524952e6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:44.755477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aba792e2ea646fb8006e74e524952e6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:44.755477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aba792e2ea646fb8006e74e524952e6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:44.776929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 916/1806] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:44 2022] POST /identity/v3/projects => generated 409 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:04:44.783400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:44.807148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:44.829041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aba792e2ea646fb8006e74e524952e6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:44.838821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6c4f389f-0755-40e8-938d-7089d61be232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aba792e2ea646fb8006e74e524952e6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:44.896763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 891/1807] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:44 2022] DELETE /identity/v3/users/7aba792e2ea646fb8006e74e524952e6 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:44.903507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a5a65-30da-4ad4-94a1-e9d5d89fc9d0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:44.905564 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a5a65-30da-4ad4-94a1-e9d5d89fc9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:44.906641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a5a65-30da-4ad4-94a1-e9d5d89fc9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:44.986579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6f4a5a65-30da-4ad4-94a1-e9d5d89fc9d0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:44.987978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:44.991197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6f4a5a65-30da-4ad4-94a1-e9d5d89fc9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:45.021779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadf25b-4e9f-4f70-bf75-69c8ce4ea971 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:45.207195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 892/1808] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:45.211879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4cc6-ed98-4e57-857c-330060af7fbd None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.212795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4cc6-ed98-4e57-857c-330060af7fbd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.220090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4cc6-ed98-4e57-857c-330060af7fbd None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.224551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 893/1809] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:04:45 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:04:45.232296 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb0a00-d39f-40d0-9f5b-420989eddbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:45.233984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0a00-d39f-40d0-9f5b-420989eddbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.234577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0a00-d39f-40d0-9f5b-420989eddbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.235581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb0a00-d39f-40d0-9f5b-420989eddbb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.236576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb0a00-d39f-40d0-9f5b-420989eddbb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:45.239871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb0a00-d39f-40d0-9f5b-420989eddbb1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:45.271218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c5e43-bc77-47d8-8d57-3ad24e70d6e1 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.272044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c5e43-bc77-47d8-8d57-3ad24e70d6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.272697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c5e43-bc77-47d8-8d57-3ad24e70d6e1 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.274707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 894/1810] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 11:04:45.300132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c5e43-bc77-47d8-8d57-3ad24e70d6e1 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:45.303968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2f9c5e43-bc77-47d8-8d57-3ad24e70d6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:45.346553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 917/1811] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:44 2022] POST /identity/v3/users => generated 311 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:45.354875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3519e9-77d7-4b03-83ea-35ca265d15dd None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.365155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3519e9-77d7-4b03-83ea-35ca265d15dd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.374996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3519e9-77d7-4b03-83ea-35ca265d15dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.457617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-3f3519e9-77d7-4b03-83ea-35ca265d15dd None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:45.459348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3f3519e9-77d7-4b03-83ea-35ca265d15dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:45.592768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c5e43-bc77-47d8-8d57-3ad24e70d6e1 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:45.598303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678bbab-9856-4d95-8570-ccf398c24039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:45.599552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bbab-9856-4d95-8570-ccf398c24039 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.599901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bbab-9856-4d95-8570-ccf398c24039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.600345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678bbab-9856-4d95-8570-ccf398c24039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.613368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678bbab-9856-4d95-8570-ccf398c24039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:45.615152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678bbab-9856-4d95-8570-ccf398c24039 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:45.628369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 895/1812] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:04:45.630062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219c57-c3b2-439a-b343-6a873fc79644 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.630448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219c57-c3b2-439a-b343-6a873fc79644 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.630785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219c57-c3b2-439a-b343-6a873fc79644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.656433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a0219c57-c3b2-439a-b343-6a873fc79644 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:45.658179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a0219c57-c3b2-439a-b343-6a873fc79644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:45.740397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 918/1813] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:45.743776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3519e9-77d7-4b03-83ea-35ca265d15dd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:45.790297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3519e9-77d7-4b03-83ea-35ca265d15dd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:45.805422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:45.806709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.807001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.807595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.809971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:45.814354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:45.859732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 919/1814] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:45.865979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:45.868073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:45.914674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 896/1815] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:45.917297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219c57-c3b2-439a-b343-6a873fc79644 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:45.928487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fb592-f9d6-4625-b523-57508d9306a7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:45.947489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219c57-c3b2-439a-b343-6a873fc79644 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:45.953387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:45.955524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.955810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.956087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] PATH_INFO: `/v3/projects/b053455a4101415fb9dd6fb8c0aec4f5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:45.964830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b053455a4101415fb9dd6fb8c0aec4f5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:45.968815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:45.973316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:45.974495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:45.974837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:45.975161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] PATH_INFO: `/v3/projects/6207a96e23954950ad534810ea49fa65/users/4cb0fec9808f4824b6a834345463007a/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:46.004677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6207a96e23954950ad534810ea49fa65, user_id=4cb0fec9808f4824b6a834345463007a, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:46.007934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:46.084453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 897/1816] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:45 2022] PUT /identity/v3/projects/6207a96e23954950ad534810ea49fa65/users/4cb0fec9808f4824b6a834345463007a/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:46.086096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:46.090485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:46.150057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:46.185031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b053455a4101415fb9dd6fb8c0aec4f5', 'request_id': 'req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.214109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b053455a4101415fb9dd6fb8c0aec4f5', 'request_id': 'req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.234214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b053455a4101415fb9dd6fb8c0aec4f5', 'request_id': 'req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.286361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb1f4b-0fd6-47a4-9397-3deeadf5ebaa admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:46.312485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] The token cache is being invalidate because project b053455a4101415fb9dd6fb8c0aec4f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:46.312781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.325205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.329754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 920/1817] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:45 2022] DELETE /identity/v3/projects/b053455a4101415fb9dd6fb8c0aec4f5 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:46.332790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:46.341269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:46.409691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:46.411934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:46.412598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:46.414880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:46.418772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:46.420859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:46.443106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:46.446711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:46.472672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 898/1818] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:46.473956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:46.481910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:46.586865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e9d9d-688c-4b67-a404-9e757ebb19fe admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:46.596838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:46.598251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:46.598844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:46.599430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] PATH_INFO: `/v3/users/b1fcb766ef9044c4badf65337a6cc7a7` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:46.601789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1fcb766ef9044c4badf65337a6cc7a7)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:46.603961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:46.673391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304f9bc-1b98-4762-9744-4e3f5a85d656 service nova] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:46.689530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:46.691797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:46.692151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:46.692498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:46.694683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:46.700147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:46.769123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 899/1819] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:46.775229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:46.779641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:46.811328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7302e01-ca1f-42bf-a349-334953b086b4 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:46.818197 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:46.818923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:46.819201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:46.819663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] PATH_INFO: `/v3/users/f65206003da6400bbb32e543e18761a1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:46.826835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65206003da6400bbb32e543e18761a1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:46.830849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:46.868918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1fcb766ef9044c4badf65337a6cc7a7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.870794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1fcb766ef9044c4badf65337a6cc7a7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.870794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1fcb766ef9044c4badf65337a6cc7a7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:46.899789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1fcb766ef9044c4badf65337a6cc7a7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:47.013000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1fcb766ef9044c4badf65337a6cc7a7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:47.073685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 921/1820] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:46 2022] DELETE /identity/v3/users/b1fcb766ef9044c4badf65337a6cc7a7 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:47.076384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:47.076384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:47.139642 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65206003da6400bbb32e543e18761a1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:47.139642 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65206003da6400bbb32e543e18761a1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:47.139642 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65206003da6400bbb32e543e18761a1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:47.233378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65206003da6400bbb32e543e18761a1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:47.245251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65206003da6400bbb32e543e18761a1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:47.255744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810d037-9ce3-4903-a224-47feb804b3cd admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:47.266569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b2126-7714-4958-908c-3be0aa42cb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:47.267270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b2126-7714-4958-908c-3be0aa42cb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:47.267557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b2126-7714-4958-908c-3be0aa42cb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:47.271833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b2126-7714-4958-908c-3be0aa42cb04 admin admin] PATH_INFO: `/v3/projects/6207a96e23954950ad534810ea49fa65/users/4cb0fec9808f4824b6a834345463007a/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:47.274382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b2126-7714-4958-908c-3be0aa42cb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6207a96e23954950ad534810ea49fa65, user_id=4cb0fec9808f4824b6a834345463007a, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:47.275088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 900/1821] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:46 2022] DELETE /identity/v3/users/f65206003da6400bbb32e543e18761a1 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:47.276147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:47.280932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b2126-7714-4958-908c-3be0aa42cb04 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:47.281096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:47.297472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 922/1822] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:47 2022] PUT /identity/v3/projects/6207a96e23954950ad534810ea49fa65/users/4cb0fec9808f4824b6a834345463007a/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:47.297804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b07eb6-95c1-48be-8381-964369006729 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:47.310374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b07eb6-95c1-48be-8381-964369006729 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:47.310374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b07eb6-95c1-48be-8381-964369006729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:47.368666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:47.385472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-46b07eb6-95c1-48be-8381-964369006729 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:47.386826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-46b07eb6-95c1-48be-8381-964369006729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:47.452940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb8161-7044-4f25-a1df-e3d09bc5c157 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:47.553333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:47.572126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 923/1823] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:47.572126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:47.572126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:47.572126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:47.616504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:47.617589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:47.618012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:47.618337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:47.620423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:47.634093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:47.635861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:47.650412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 901/1824] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:47.651178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:47.656098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:47.656219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] MFA Rules not processed for user `4cb0fec9808f4824b6a834345463007a`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:47.656219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:47.762212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 924/1825] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:47 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:47.763957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:47.765153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:47.775426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55717263-ed28-4264-bb0d-144066b759f7 service nova] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:47.784616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:47.785210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:47.785491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:47.785905 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:47.799903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:47.802417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:47.815986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 902/1826] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:47.837830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7ae65-c218-4c81-a3ef-20bc3b1de546 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:47.842220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:47.842796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:47.843060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:47.843330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:47.855434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:47.856910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:47.876229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 925/1827] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:47 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:48.348854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:48.350984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:48.373056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e01aca-52f1-4aa2-9eee-22330087c2fa service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:48.384352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:48.385454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:48.392477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:48.392986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] PATH_INFO: `/v3/projects/be3ee0b1d7e347c6b9e3ab3dd6c75428` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:48.394784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3ee0b1d7e347c6b9e3ab3dd6c75428)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:48.396719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:48.541694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:48.553411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925c86f-a35e-439b-b3eb-fe1f585873ea service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:48.579838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3ee0b1d7e347c6b9e3ab3dd6c75428', 'request_id': 'req-8c2a3624-21b6-4668-a759-af9b8e27ef06'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:48.598448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3ee0b1d7e347c6b9e3ab3dd6c75428', 'request_id': 'req-8c2a3624-21b6-4668-a759-af9b8e27ef06'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:48.609399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3ee0b1d7e347c6b9e3ab3dd6c75428', 'request_id': 'req-8c2a3624-21b6-4668-a759-af9b8e27ef06'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:48.621834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:48.646964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:48.647721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:48.648051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:48.648435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:48.656148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:48.745054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:48.746272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] The token cache is being invalidate because project be3ee0b1d7e347c6b9e3ab3dd6c75428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:48.746521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:48.746932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:48.747771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:48.751940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 903/1828] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:48 2022] DELETE /identity/v3/projects/be3ee0b1d7e347c6b9e3ab3dd6c75428 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:48.756784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:48.757131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 926/1829] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:48.760831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:48.765353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:48.962410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a3624-21b6-4668-a759-af9b8e27ef06 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:48.978638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:48.979423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:48.979760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:48.980167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] PATH_INFO: `/v3/users/6209170192a34ea2b9483cf784022d99` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:48.983244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6209170192a34ea2b9483cf784022d99)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:48.986088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:49.046712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:49.052612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:49.090985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1ce7a-c8cb-4504-9c78-4355b839fa4a service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:49.131886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:49.135399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:49.137445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:49.139287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] PATH_INFO: `/v3/projects/35111c1393a94883972f8002325504aa` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:49.146949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35111c1393a94883972f8002325504aa)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:49.157376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:49.216363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6209170192a34ea2b9483cf784022d99'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.217315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6209170192a34ea2b9483cf784022d99'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.218262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6209170192a34ea2b9483cf784022d99'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.304950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6209170192a34ea2b9483cf784022d99'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.324355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b85b9d2-1d61-40c4-aef5-3f4b917224ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6209170192a34ea2b9483cf784022d99'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.372654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35111c1393a94883972f8002325504aa', 'request_id': 'req-a6ffb469-6fbf-458f-893b-422c310e8998'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.379253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35111c1393a94883972f8002325504aa', 'request_id': 'req-a6ffb469-6fbf-458f-893b-422c310e8998'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.380155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 904/1830] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:48 2022] DELETE /identity/v3/users/6209170192a34ea2b9483cf784022d99 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:49.391129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35111c1393a94883972f8002325504aa', 'request_id': 'req-a6ffb469-6fbf-458f-893b-422c310e8998'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.515699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] The token cache is being invalidate because project 35111c1393a94883972f8002325504aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:49.520587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.520587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a6ffb469-6fbf-458f-893b-422c310e8998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:49.520587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 927/1831] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:49 2022] DELETE /identity/v3/projects/35111c1393a94883972f8002325504aa => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:49.531322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e1ca-ee8b-4a0c-9b65-789a68b433a2 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:49.531822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e1ca-ee8b-4a0c-9b65-789a68b433a2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:49.532100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e1ca-ee8b-4a0c-9b65-789a68b433a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:49.591234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-bc09e1ca-ee8b-4a0c-9b65-789a68b433a2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:49.594356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bc09e1ca-ee8b-4a0c-9b65-789a68b433a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:49.740225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 905/1832] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:49.752454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599366a7-32c7-4eae-8d4e-3813a4fc025d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:49.752984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599366a7-32c7-4eae-8d4e-3813a4fc025d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:49.752984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599366a7-32c7-4eae-8d4e-3813a4fc025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:49.805344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-599366a7-32c7-4eae-8d4e-3813a4fc025d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:49.806605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-599366a7-32c7-4eae-8d4e-3813a4fc025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:49.840469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 928/1833] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:49.842994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e1ca-ee8b-4a0c-9b65-789a68b433a2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:49.895627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e1ca-ee8b-4a0c-9b65-789a68b433a2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:49.925785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:49.930971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:49.931298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:49.933500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:49.937397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:49.937727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:49.954273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:49.988693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 906/1834] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:49.993804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599366a7-32c7-4eae-8d4e-3813a4fc025d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:50.008387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599366a7-32c7-4eae-8d4e-3813a4fc025d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:50.014381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:50.016215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:50.016706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:50.017205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:50.018305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:50.021322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:50.138039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 929/1835] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:49 2022] POST /identity/v3/projects => generated 361 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:04:50.140548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:50.165250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431c965-aacc-4f15-a1a8-0a0cb1e8c8d5 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:50.183228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:50.274850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:50.329858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:50.332872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:50.443980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:50.448181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:50.452165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:50.452599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:50.453967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:50.457077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:50.466533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e479c1-034c-4086-b421-642d38279900 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:50.475738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:50.477104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:50.477838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:50.478180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] PATH_INFO: `/v3/projects/1fec90bd753a4dd2a61314818fda6255` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:50.479578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fec90bd753a4dd2a61314818fda6255)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:50.481861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:50.556056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 907/1836] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:50 2022] POST /identity/v3/users => generated 281 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:04:50.559763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:50.576940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917cd34-9d87-4dd7-b003-6c5aac142cad admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:50.586205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:50.586205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:50.586205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:50.586205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:50.596226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:50.598284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:50.623498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 908/1837] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:50.643249 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:50.678785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c5e68-c52f-4092-ad31-c74ebe0b0697 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:50.687923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fec90bd753a4dd2a61314818fda6255', 'request_id': 'req-b3321dac-78a4-4809-a91d-62b2a716050f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:50.711656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fec90bd753a4dd2a61314818fda6255', 'request_id': 'req-b3321dac-78a4-4809-a91d-62b2a716050f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:50.731036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fec90bd753a4dd2a61314818fda6255', 'request_id': 'req-b3321dac-78a4-4809-a91d-62b2a716050f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:50.831007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] The token cache is being invalidate because project 1fec90bd753a4dd2a61314818fda6255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:50.831648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:50.832077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:50.845707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 930/1838] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:50 2022] DELETE /identity/v3/projects/1fec90bd753a4dd2a61314818fda6255 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:50.852402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:50.870825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:50.903309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:50.904895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:50.910173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:50.910715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:50.916428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:50.917423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:50.920912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:50.949678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 909/1839] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:50.959482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:50.962349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:50.988620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13260a-bb2c-4e22-894c-f37f6cd9099c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:51.000714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ceed5-8910-44b4-8855-47b3ea99948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:51.002271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ceed5-8910-44b4-8855-47b3ea99948e admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:51.003786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ceed5-8910-44b4-8855-47b3ea99948e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:51.003786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ceed5-8910-44b4-8855-47b3ea99948e admin admin] PATH_INFO: `/v3/projects/83906260d9bf454d922dd501f33ef336/users/df7973ad052d49279fb6f662c22f6afe/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:51.044303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ceed5-8910-44b4-8855-47b3ea99948e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83906260d9bf454d922dd501f33ef336, user_id=df7973ad052d49279fb6f662c22f6afe, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:51.058278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3321dac-78a4-4809-a91d-62b2a716050f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:51.059755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ceed5-8910-44b4-8855-47b3ea99948e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:51.075941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:51.076960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:51.077338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:51.077779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] PATH_INFO: `/v3/users/5ff3d22c3e194dc8987770e5a55c41ae` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:51.079093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff3d22c3e194dc8987770e5a55c41ae)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:51.081261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:51.131818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 910/1840] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:50 2022] PUT /identity/v3/projects/83906260d9bf454d922dd501f33ef336/users/df7973ad052d49279fb6f662c22f6afe/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:51.134613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:51.137665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:51.137665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:51.171714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] MFA Rules not processed for user `df7973ad052d49279fb6f662c22f6afe`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:51.174319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:51.346484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 911/1841] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:51.409316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:51.416157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:51.470100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff3d22c3e194dc8987770e5a55c41ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:51.471301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff3d22c3e194dc8987770e5a55c41ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:51.471974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff3d22c3e194dc8987770e5a55c41ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:51.482670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff3d22c3e194dc8987770e5a55c41ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:51.571837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff3d22c3e194dc8987770e5a55c41ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:51.574579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bad99-beef-4a31-bc19-10c58789628e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:51.578938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:51.579599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:51.579884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:51.580227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:51.598788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:51.619499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:51.621150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:51.631941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 931/1842] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:50 2022] DELETE /identity/v3/users/5ff3d22c3e194dc8987770e5a55c41ae => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:51.654949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 912/1843] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:51.673136 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:51.675044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:51.769335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:51.770862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:51.795130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe7761-8194-4dc4-a870-c3e249a0e70c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:51.799076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:51.799914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:51.800280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:51.800765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] PATH_INFO: `/v3/users/df7973ad052d49279fb6f662c22f6afe` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:51.801706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7973ad052d49279fb6f662c22f6afe)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:51.806485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:51.920617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:51.952832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8131a-d349-4550-aa04-14e3f9b90ba8 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:51.962696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:51.963446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:51.963775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:51.964047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] PATH_INFO: `/v3/projects/388ddb2ce7a74666afaba1b140a4b081` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:51.965246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] RBAC: Authorizing `identity:delete_project(project_id=388ddb2ce7a74666afaba1b140a4b081)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:51.967310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.113373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7973ad052d49279fb6f662c22f6afe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.116982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7973ad052d49279fb6f662c22f6afe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.117627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7973ad052d49279fb6f662c22f6afe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.130051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388ddb2ce7a74666afaba1b140a4b081', 'request_id': 'req-195b8447-9fa9-4bd1-a927-aab52c30313c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.137347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7973ad052d49279fb6f662c22f6afe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.147383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388ddb2ce7a74666afaba1b140a4b081', 'request_id': 'req-195b8447-9fa9-4bd1-a927-aab52c30313c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.156388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388ddb2ce7a74666afaba1b140a4b081', 'request_id': 'req-195b8447-9fa9-4bd1-a927-aab52c30313c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.211018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7196e613-1e74-4d5f-969b-3369720aa607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7973ad052d49279fb6f662c22f6afe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.237949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 932/1844] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:51 2022] DELETE /identity/v3/users/df7973ad052d49279fb6f662c22f6afe => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:52.239938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.245729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.245729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.263455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:52.268649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.293254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] The token cache is being invalidate because project 388ddb2ce7a74666afaba1b140a4b081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:52.294081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.297325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-195b8447-9fa9-4bd1-a927-aab52c30313c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.297325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 913/1845] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:51 2022] DELETE /identity/v3/projects/388ddb2ce7a74666afaba1b140a4b081 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:52.301302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfd95-0a8d-4338-bccd-a9e4cda6fcd0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.302031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfd95-0a8d-4338-bccd-a9e4cda6fcd0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.302031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dfd95-0a8d-4338-bccd-a9e4cda6fcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.331723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-770dfd95-0a8d-4338-bccd-a9e4cda6fcd0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:52.331723 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-770dfd95-0a8d-4338-bccd-a9e4cda6fcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.340494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 933/1846] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:52.359768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.361841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.399147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 914/1847] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:52.406675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.406958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.407228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.426850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c26bb3-ccd6-41c7-ba9e-10a43d85ea45 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.428118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:52.429615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.430915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.431242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.431524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.431792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.442408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 915/1848] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:52.442408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.442408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.442757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.443764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.454581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd822841-b778-469a-81ad-4ec5e83071cf None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.457187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 934/1849] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:04:52.458097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.462098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.463063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.463334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.464042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] PATH_INFO: `/v3/projects/a0ebde3397b04716a49043896ff32827` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.465594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ebde3397b04716a49043896ff32827)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.467357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.468962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c6a0f-257d-479b-b2f3-6eefc2d34540 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.474459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.475064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.475392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.475941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.476645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:52.477338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.478635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.494752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 935/1850] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:52.495812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.498388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:52.508551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ee689-97a6-4de3-a70e-d166d1eff766 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.513473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.514268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.514649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.514933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.515629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.517536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.593036 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ebde3397b04716a49043896ff32827', 'request_id': 'req-8449cc9b-7420-436d-a77c-a121365ea8f6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.598578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ebde3397b04716a49043896ff32827', 'request_id': 'req-8449cc9b-7420-436d-a77c-a121365ea8f6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.606217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ebde3397b04716a49043896ff32827', 'request_id': 'req-8449cc9b-7420-436d-a77c-a121365ea8f6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.607802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 936/1851] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:04:52.612436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.684409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] The token cache is being invalidate because project a0ebde3397b04716a49043896ff32827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:52.684842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.686256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8449cc9b-7420-436d-a77c-a121365ea8f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:52.694673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 916/1852] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:52 2022] DELETE /identity/v3/projects/a0ebde3397b04716a49043896ff32827 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:52.696523 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db53f6d-7be6-47bc-8ebe-21488dceee60 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.696818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db53f6d-7be6-47bc-8ebe-21488dceee60 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.697151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db53f6d-7be6-47bc-8ebe-21488dceee60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.702322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c9014-88c5-4193-b420-9b58c390858f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.708019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.710921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.710921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.710921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.710921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.713185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.722872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3db53f6d-7be6-47bc-8ebe-21488dceee60 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:52.724202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3db53f6d-7be6-47bc-8ebe-21488dceee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.744492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 917/1853] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:52.748593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591abeb-add0-4f74-b9a6-502d2a779d7e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.749037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591abeb-add0-4f74-b9a6-502d2a779d7e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.749481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591abeb-add0-4f74-b9a6-502d2a779d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.773753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3591abeb-add0-4f74-b9a6-502d2a779d7e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:52.776168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3591abeb-add0-4f74-b9a6-502d2a779d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.798616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 918/1854] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:52.799617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591abeb-add0-4f74-b9a6-502d2a779d7e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.810122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591abeb-add0-4f74-b9a6-502d2a779d7e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.815171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.815904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.816283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.816656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.817408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:52.817800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.829642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 937/1855] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:04:52.837670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.838924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.841010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:52.858028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88634234-d348-462c-b087-6bce67a79841 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.860750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 919/1856] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:52.869930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.869930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.869930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.869930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.870380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.875680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.892113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.900459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73387-90a3-4144-95ae-81f0c26cf202 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.906566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.907402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.907781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.908300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.908944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.911766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.931096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 938/1857] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:04:52.941500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.953684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02d7c3-b342-406b-b619-e1c3e8c47ca6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:52.959009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:52.959618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:52.959898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:52.960780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:52.961496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:52.963189 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:52.975265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 939/1858] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:04:52.979412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:52.990443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237fd74-136c-45b6-b64e-cd02030fba25 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.001479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.001479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.001479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.001479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] PATH_INFO: `/v3/projects/ba979583cdf94a31a03eec35f48b808b/users/cd2cb172778b42a1ac2ba46f9937cf52/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.011460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba979583cdf94a31a03eec35f48b808b, user_id=cd2cb172778b42a1ac2ba46f9937cf52, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.014091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.030348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 920/1859] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:52 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:53.039609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.039609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:53.103713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 940/1860] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:52 2022] PUT /identity/v3/projects/ba979583cdf94a31a03eec35f48b808b/users/cd2cb172778b42a1ac2ba46f9937cf52/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:53.104437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b641-5aa1-4da9-9cec-9229541cb156 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.105139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.171454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.172464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.172936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.173443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] PATH_INFO: `/v3/projects/83906260d9bf454d922dd501f33ef336` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.174818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83906260d9bf454d922dd501f33ef336)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.176994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.179680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cee81-c791-41f0-a928-73b83323e553 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.184638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b7a92-fb62-4741-848c-33e576785eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.185246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b7a92-fb62-4741-848c-33e576785eea admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.185529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b7a92-fb62-4741-848c-33e576785eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.185801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b7a92-fb62-4741-848c-33e576785eea admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.186480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b7a92-fb62-4741-848c-33e576785eea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.188376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b7a92-fb62-4741-848c-33e576785eea admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.246633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 941/1861] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:53.248042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c829c-1ab6-4d58-82ba-30bd65444d88 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.248329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c829c-1ab6-4d58-82ba-30bd65444d88 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.248673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c829c-1ab6-4d58-82ba-30bd65444d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.274072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83906260d9bf454d922dd501f33ef336', 'request_id': 'req-c149fb7a-2802-43f9-a9f0-2a61861b0e60'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:53.276548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-151c829c-1ab6-4d58-82ba-30bd65444d88 None None] MFA Rules not processed for user `cd2cb172778b42a1ac2ba46f9937cf52`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:53.277765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-151c829c-1ab6-4d58-82ba-30bd65444d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:53.279945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83906260d9bf454d922dd501f33ef336', 'request_id': 'req-c149fb7a-2802-43f9-a9f0-2a61861b0e60'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:53.286074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83906260d9bf454d922dd501f33ef336', 'request_id': 'req-c149fb7a-2802-43f9-a9f0-2a61861b0e60'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:53.342625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 942/1862] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:53.343630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c829c-1ab6-4d58-82ba-30bd65444d88 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.364978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] The token cache is being invalidate because project 83906260d9bf454d922dd501f33ef336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:53.365485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:53.366919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:53.368904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 921/1863] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:53 2022] DELETE /identity/v3/projects/83906260d9bf454d922dd501f33ef336 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:53.369571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.370978 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:53.377347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:53.410790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c829c-1ab6-4d58-82ba-30bd65444d88 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.415581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882f020-0f1f-4e17-8e2d-0044502d021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.416196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882f020-0f1f-4e17-8e2d-0044502d021d admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.416485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882f020-0f1f-4e17-8e2d-0044502d021d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.416799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882f020-0f1f-4e17-8e2d-0044502d021d admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.417455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882f020-0f1f-4e17-8e2d-0044502d021d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.419075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882f020-0f1f-4e17-8e2d-0044502d021d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.433558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 943/1864] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:53.433558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.433558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.433558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.444517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149fb7a-2802-43f9-a9f0-2a61861b0e60 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.450117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.450770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.451050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.451811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.451948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.454371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.458462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:53.458462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:53.470945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 944/1865] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:53.470945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.472839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:53.485728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02575f12-e7db-462a-b4e8-5351816197eb None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.490502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9180a-3a43-4e61-82cd-f1fc1a901cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.491214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9180a-3a43-4e61-82cd-f1fc1a901cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.491569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9180a-3a43-4e61-82cd-f1fc1a901cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.492353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9180a-3a43-4e61-82cd-f1fc1a901cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.492487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9180a-3a43-4e61-82cd-f1fc1a901cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.494095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9180a-3a43-4e61-82cd-f1fc1a901cc9 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.507578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 945/1866] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:53.507578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6caa0e-a2ba-46c6-baa1-a9f9f7972b26 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.507578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6caa0e-a2ba-46c6-baa1-a9f9f7972b26 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.507578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6caa0e-a2ba-46c6-baa1-a9f9f7972b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.526357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6c6caa0e-a2ba-46c6-baa1-a9f9f7972b26 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:53.527623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6c6caa0e-a2ba-46c6-baa1-a9f9f7972b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:53.536032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 922/1867] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:53.536032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.539867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 946/1868] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:53.540609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6caa0e-a2ba-46c6-baa1-a9f9f7972b26 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.550065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6caa0e-a2ba-46c6-baa1-a9f9f7972b26 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.612527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.613137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.613416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.613721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.614382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.614734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a1099-40df-44bd-bb0f-8642441f29dc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.616410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.623926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.624613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.624891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.625161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] PATH_INFO: `/v3/projects/af7e2d5bc411489d9a58e5638a2f854e/users/49ba9a1c10e3468c962379ffd58dd434/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.643432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7e2d5bc411489d9a58e5638a2f854e, user_id=49ba9a1c10e3468c962379ffd58dd434, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.646106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.724902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 947/1869] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:53.726415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.736526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b39e1f-9322-4657-9e9b-20ebf000c206 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.736865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 923/1870] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:53 2022] PUT /identity/v3/projects/af7e2d5bc411489d9a58e5638a2f854e/users/49ba9a1c10e3468c962379ffd58dd434/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:53.737681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.793346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:53.813394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af9c2f-85a0-4ea3-bbc5-d9c18a826ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.814568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9c2f-85a0-4ea3-bbc5-d9c18a826ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.816828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9c2f-85a0-4ea3-bbc5-d9c18a826ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.817132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af9c2f-85a0-4ea3-bbc5-d9c18a826ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.818018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af9c2f-85a0-4ea3-bbc5-d9c18a826ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:53.818493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af9c2f-85a0-4ea3-bbc5-d9c18a826ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.819256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957ecc4-1ea6-43cd-89e9-672ff84ca78f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.821110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af9c2f-85a0-4ea3-bbc5-d9c18a826ad2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.827127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.827127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.829430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.829430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.829430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.830876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.836871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 948/1871] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:53.838783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf3f4b-a00a-48aa-897e-c8a33e98a5c3 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.839223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf3f4b-a00a-48aa-897e-c8a33e98a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.839642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf3f4b-a00a-48aa-897e-c8a33e98a5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.849696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 924/1872] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:53.849696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:53.861563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8ed23-80f9-4fe1-b7c4-6aa7c14645e9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:53.871965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:53.872764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:53.873044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:53.873317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:53.873951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:53.876395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:53.893361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-16bf3f4b-a00a-48aa-897e-c8a33e98a5c3 None None] MFA Rules not processed for user `49ba9a1c10e3468c962379ffd58dd434`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:53.894661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-16bf3f4b-a00a-48aa-897e-c8a33e98a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:53.996792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 949/1873] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:53.997744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf3f4b-a00a-48aa-897e-c8a33e98a5c3 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.010359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf3f4b-a00a-48aa-897e-c8a33e98a5c3 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.020285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 925/1874] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:53 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:54.020285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.068460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:54.081291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.082029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.082354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.084443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] PATH_INFO: `/v3/projects/8ea8d4f0f7cd49f3ba966a0938cb67e2/users/4056cad5472445ebb85f01c49df9fb0d/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.084563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5274d5-45ee-40c1-b4bd-7222920bdf11 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.088704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.089331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.089620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.089886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.090547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.092708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.099401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea8d4f0f7cd49f3ba966a0938cb67e2, user_id=4056cad5472445ebb85f01c49df9fb0d, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.103142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.188066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 950/1875] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:53 2022] PUT /identity/v3/projects/8ea8d4f0f7cd49f3ba966a0938cb67e2/users/4056cad5472445ebb85f01c49df9fb0d/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:54.188816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.210057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 926/1876] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:54 2022] POST /identity/v3/projects => generated 367 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:54.210057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.255017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae0151-5018-4de2-8030-098603c82f02 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.279307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ba261-627f-4e39-9022-291f09530988 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.286062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.286844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.287229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.287602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.289867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.291652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.302352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 927/1877] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:54.303335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.315351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37ae0-1740-4c73-a663-377980e76fdb admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.321784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.322559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.322885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.323099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.323552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.323690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.323973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.324242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.324545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.324869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.326822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.327105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.421641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 928/1878] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:54 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:54.422457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.432816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830c96c-d3d0-4c0f-b4d8-7a366b1070c4 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.438627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.439284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.439703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.440078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] PATH_INFO: `/v3/projects/8ea8d4f0f7cd49f3ba966a0938cb67e2/users/4056cad5472445ebb85f01c49df9fb0d/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.442507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea8d4f0f7cd49f3ba966a0938cb67e2, user_id=4056cad5472445ebb85f01c49df9fb0d, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.445510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.446200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 951/1879] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:54 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:54.446960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.458054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 929/1880] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:54 2022] PUT /identity/v3/projects/8ea8d4f0f7cd49f3ba966a0938cb67e2/users/4056cad5472445ebb85f01c49df9fb0d/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:54.458906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec0feb-b51f-412a-b256-18db68d3ec0a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.459033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.516698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2125c8-69ca-44f9-bbeb-71284b8e1bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.517414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2125c8-69ca-44f9-bbeb-71284b8e1bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.517694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2125c8-69ca-44f9-bbeb-71284b8e1bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.518716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2125c8-69ca-44f9-bbeb-71284b8e1bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.518716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2125c8-69ca-44f9-bbeb-71284b8e1bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.520234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2125c8-69ca-44f9-bbeb-71284b8e1bf4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.520695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda90ea-c5b2-4be6-8361-8f6d2bf437ee admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.525862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.526981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.527359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.527657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.528373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.530208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.531191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 952/1881] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:54.532421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66564b9-4a46-4ade-9c34-5b51cb7042ab None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.532712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66564b9-4a46-4ade-9c34-5b51cb7042ab None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.532990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66564b9-4a46-4ade-9c34-5b51cb7042ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.543401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 930/1882] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:54.544298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.556593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d252de0-2b64-4d1f-8599-ccba40b45d16 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.562192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.562879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.563280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.563660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] PATH_INFO: `/v3/projects/22010ba4f41f4b26ab0cf6471d5267a1/users/7544f00e764e4779af1fa6802450ea04/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.564092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e66564b9-4a46-4ade-9c34-5b51cb7042ab None None] MFA Rules not processed for user `4056cad5472445ebb85f01c49df9fb0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:54.565257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e66564b9-4a46-4ade-9c34-5b51cb7042ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:54.581044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22010ba4f41f4b26ab0cf6471d5267a1, user_id=7544f00e764e4779af1fa6802450ea04, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.583972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.641719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 953/1883] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:54.644001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66564b9-4a46-4ade-9c34-5b51cb7042ab None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.657964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66564b9-4a46-4ade-9c34-5b51cb7042ab None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.662829 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.664038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.664303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.664568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.665217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.667026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.677404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 954/1884] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:54.678252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.681146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:54.740563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ec47d-d674-41ab-8b37-73413f35ee47 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.745559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.746179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.746509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.746783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.756363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.757863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:54.767999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 955/1885] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:54.768648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:54.782185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d15b60-412a-40f8-9fb1-bf1808de3085 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:54.791984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:54.792883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:54.793272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:54.793609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] PATH_INFO: `/v3/projects/f677f169706a426d8e3efd823f29dbdb/users/e44727cbe0d9471cbf788127d82b606b/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:54.813872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f677f169706a426d8e3efd823f29dbdb, user_id=e44727cbe0d9471cbf788127d82b606b, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:54.816697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:55.625598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 931/1886] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:54 2022] PUT /identity/v3/projects/22010ba4f41f4b26ab0cf6471d5267a1/users/7544f00e764e4779af1fa6802450ea04/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 1081 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:55.628072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:55.646810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:55.655129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 956/1887] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:54 2022] PUT /identity/v3/projects/f677f169706a426d8e3efd823f29dbdb/users/e44727cbe0d9471cbf788127d82b606b/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 887 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:55.662425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:55.691944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f11b2-4876-4e8c-af12-d1209cce5967 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:55.722645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898f5f-2714-4c8d-ab4b-b4fc383077ac admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:55.728429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5176b3e3-1c11-4d64-ba4b-dc3963fa1411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:55.729242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b3e3-1c11-4d64-ba4b-dc3963fa1411 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:55.729540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b3e3-1c11-4d64-ba4b-dc3963fa1411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:55.729817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5176b3e3-1c11-4d64-ba4b-dc3963fa1411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:55.730500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176b3e3-1c11-4d64-ba4b-dc3963fa1411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:55.737241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5176b3e3-1c11-4d64-ba4b-dc3963fa1411 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:55.746048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 957/1888] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:55.746048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c7a03-e05c-4493-914e-4261770480a7 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:55.746048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c7a03-e05c-4493-914e-4261770480a7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:55.746048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c7a03-e05c-4493-914e-4261770480a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:55.752856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:55.753640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:55.754100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:55.754680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] PATH_INFO: `/v3/users/cd2cb172778b42a1ac2ba46f9937cf52` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:55.756014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2cb172778b42a1ac2ba46f9937cf52)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:55.758724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:55.776099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-740c7a03-e05c-4493-914e-4261770480a7 None None] MFA Rules not processed for user `e44727cbe0d9471cbf788127d82b606b`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:55.777590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-740c7a03-e05c-4493-914e-4261770480a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:55.849810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 958/1889] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:55.851305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c7a03-e05c-4493-914e-4261770480a7 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:55.869562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c7a03-e05c-4493-914e-4261770480a7 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:55.874349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:55.875049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:55.875330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:55.875604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] PATH_INFO: `/v3/projects/22010ba4f41f4b26ab0cf6471d5267a1/users/7544f00e764e4779af1fa6802450ea04/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:55.877751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22010ba4f41f4b26ab0cf6471d5267a1, user_id=7544f00e764e4779af1fa6802450ea04, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:55.880408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:55.893048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 959/1890] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:55 2022] PUT /identity/v3/projects/22010ba4f41f4b26ab0cf6471d5267a1/users/7544f00e764e4779af1fa6802450ea04/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:55.893802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:55.895429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:55.971704 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fa958-33fd-4da3-9bca-241f05fe1618 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:55.975572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c0523-5b6a-4e28-bbb1-33f6bd28de1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:55.976903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c0523-5b6a-4e28-bbb1-33f6bd28de1b service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:55.977187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c0523-5b6a-4e28-bbb1-33f6bd28de1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:55.977482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c0523-5b6a-4e28-bbb1-33f6bd28de1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:56.027369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:56.036633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c0523-5b6a-4e28-bbb1-33f6bd28de1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:56.041469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c0523-5b6a-4e28-bbb1-33f6bd28de1b service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:56.047970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 960/1891] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:56.049255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767f845-162d-4aa9-8882-c7554249c94d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:56.049536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767f845-162d-4aa9-8882-c7554249c94d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:56.049841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767f845-162d-4aa9-8882-c7554249c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:56.076703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2cb172778b42a1ac2ba46f9937cf52'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:56.077645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2cb172778b42a1ac2ba46f9937cf52'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:56.078395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2cb172778b42a1ac2ba46f9937cf52'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:56.080755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b767f845-162d-4aa9-8882-c7554249c94d None None] MFA Rules not processed for user `7544f00e764e4779af1fa6802450ea04`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:56.082737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b767f845-162d-4aa9-8882-c7554249c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:56.169993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 961/1892] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:56.174913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2cb172778b42a1ac2ba46f9937cf52'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:56.185549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2cb172778b42a1ac2ba46f9937cf52'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:56.222065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 932/1893] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:55 2022] DELETE /identity/v3/users/cd2cb172778b42a1ac2ba46f9937cf52 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:56.223797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:56.224088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:56.225274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:56.263656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:56.265169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:56.339108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 962/1894] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:56.349146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:56.350737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:56.413552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11fa84-e7bb-46ff-99d2-0247a2ea4dc8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:56.417220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:56.417812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:56.418214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:56.418514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:56.427586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:56.430785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:56.439125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 933/1895] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:56.455168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:56.456631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:56.514706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994e332-6e9f-40dd-9def-d8cb9673f3ff None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:56.519564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:56.520179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:56.520448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:56.520719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:56.578537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:56.579971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:56.589872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 963/1896] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:04:56.999688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:57.001471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:57.015760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18078-e52d-4c7b-bab4-2e77b6769a08 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:57.060571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:57.061280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:57.061280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:57.061280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:57.061280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] PATH_INFO: `/v3/projects/ba979583cdf94a31a03eec35f48b808b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:57.061848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba979583cdf94a31a03eec35f48b808b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:57.061848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:57.062224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cd7f8-54ee-4c58-bde0-5125e40b84e4 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:57.062224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:57.062224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:57.062224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:57.062579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:57.062579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:57.131687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:57.142390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:57.143275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:57.164784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 964/1897] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:04:57.189936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba979583cdf94a31a03eec35f48b808b', 'request_id': 'req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.196334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba979583cdf94a31a03eec35f48b808b', 'request_id': 'req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.203341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba979583cdf94a31a03eec35f48b808b', 'request_id': 'req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.298115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] The token cache is being invalidate because project ba979583cdf94a31a03eec35f48b808b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:57.299284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.299637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.301986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 934/1898] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:56 2022] DELETE /identity/v3/projects/ba979583cdf94a31a03eec35f48b808b => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:57.307652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:57.319057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7797cb7a-c043-4138-a23c-02be1bb72407 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:57.375140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:57.375799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:57.376076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:57.376372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] PATH_INFO: `/v3/users/4056cad5472445ebb85f01c49df9fb0d` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:57.377504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4056cad5472445ebb85f01c49df9fb0d)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:57.379422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:57.558639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4056cad5472445ebb85f01c49df9fb0d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.559330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4056cad5472445ebb85f01c49df9fb0d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.560019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4056cad5472445ebb85f01c49df9fb0d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.567535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4056cad5472445ebb85f01c49df9fb0d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.684525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d59b31c2-3173-47fc-8b92-357bc3279639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4056cad5472445ebb85f01c49df9fb0d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:57.702779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 965/1899] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:04:57 2022] DELETE /identity/v3/users/4056cad5472445ebb85f01c49df9fb0d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:58.124129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:58.322112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:58.323941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:58.343008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0ce33-5ad1-45fa-9d6e-65a5acb88a7c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:58.405768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:58.406981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:58.407494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:58.407778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] PATH_INFO: `/v3/projects/8ea8d4f0f7cd49f3ba966a0938cb67e2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:58.408932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea8d4f0f7cd49f3ba966a0938cb67e2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:58.410907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:58.549817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea8d4f0f7cd49f3ba966a0938cb67e2', 'request_id': 'req-ebd51e52-000e-459c-a700-c1bbe587ecaa'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:58.555833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea8d4f0f7cd49f3ba966a0938cb67e2', 'request_id': 'req-ebd51e52-000e-459c-a700-c1bbe587ecaa'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:58.562028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea8d4f0f7cd49f3ba966a0938cb67e2', 'request_id': 'req-ebd51e52-000e-459c-a700-c1bbe587ecaa'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:58.643895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] The token cache is being invalidate because project 8ea8d4f0f7cd49f3ba966a0938cb67e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:04:58.644191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:58.644821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ebd51e52-000e-459c-a700-c1bbe587ecaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:04:58.646679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 935/1900] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:04:58 2022] DELETE /identity/v3/projects/8ea8d4f0f7cd49f3ba966a0938cb67e2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:04:58.651834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c746-1bf7-4d2b-ae8b-46cf44153839 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:58.652343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c746-1bf7-4d2b-ae8b-46cf44153839 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:58.652434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c746-1bf7-4d2b-ae8b-46cf44153839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:58.675637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-29f1c746-1bf7-4d2b-ae8b-46cf44153839 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:58.677785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-29f1c746-1bf7-4d2b-ae8b-46cf44153839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:58.764112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 966/1901] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:58.772140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd09e72-5060-44cf-ab9e-88cfcdc62151 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:58.772476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd09e72-5060-44cf-ab9e-88cfcdc62151 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:58.777995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd09e72-5060-44cf-ab9e-88cfcdc62151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:58.799332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:04:58.802811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-1fd09e72-5060-44cf-ab9e-88cfcdc62151 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:58.805844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1fd09e72-5060-44cf-ab9e-88cfcdc62151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:58.819930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 936/1902] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:04:58.824470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1c746-1bf7-4d2b-ae8b-46cf44153839 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:58.840790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1c746-1bf7-4d2b-ae8b-46cf44153839 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:58.852048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:58.855798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:58.856081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:58.856353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:58.857028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:04:58.857325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:58.868179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:58.892987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 967/1903] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:04:58.893334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd09e72-5060-44cf-ab9e-88cfcdc62151 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:58.903419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd09e72-5060-44cf-ab9e-88cfcdc62151 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:58.908255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:58.908949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:58.909236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:58.909749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:58.910578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:58.912695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:59.017558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 937/1904] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:58 2022] POST /identity/v3/projects => generated 399 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:59.018938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:59.031940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5567925-50e4-4779-80cf-c6848891953d admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:59.119673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:59.120555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:59.123655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:59.123655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:59.123655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:59.124404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:59.215486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:59.216547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 968/1905] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:59 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:04:59.227715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5df3dd-55a9-4fab-a970-f34ce12d6f82 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:59.233566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:59.234470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:59.234609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:59.236951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:59.237637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:59.239297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:59.251813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 938/1906] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:59.255967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:59.282783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164aecf-c023-4d1e-8c6c-9ba62bdc6353 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:59.291850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe4ed9-49fe-4fda-8e47-44aab90cb830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:59.292521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe4ed9-49fe-4fda-8e47-44aab90cb830 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:59.292972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe4ed9-49fe-4fda-8e47-44aab90cb830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:59.293249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe4ed9-49fe-4fda-8e47-44aab90cb830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:59.293932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe4ed9-49fe-4fda-8e47-44aab90cb830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:59.295972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe4ed9-49fe-4fda-8e47-44aab90cb830 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:59.313643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:59.315038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 969/1907] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:04:59.325818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32563878-f967-449b-93b5-39f8241f2a7f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:59.332032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:59.332936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:59.333361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:59.333778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] PATH_INFO: `/v3/projects/c421eec5f3984c309286c7cf3a23700d/users/bbccd9c7036c4685a33a6af82ba3e344/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:59.358295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c421eec5f3984c309286c7cf3a23700d, user_id=bbccd9c7036c4685a33a6af82ba3e344, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:59.359234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:59.469704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 939/1908] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:04:59 2022] PUT /identity/v3/projects/c421eec5f3984c309286c7cf3a23700d/users/bbccd9c7036c4685a33a6af82ba3e344/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:04:59.482970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897de405-6b4d-4333-81ff-d3add2f438cc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:59.483350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897de405-6b4d-4333-81ff-d3add2f438cc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:59.484670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897de405-6b4d-4333-81ff-d3add2f438cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:59.558980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-897de405-6b4d-4333-81ff-d3add2f438cc None None] MFA Rules not processed for user `bbccd9c7036c4685a33a6af82ba3e344`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:04:59.560229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-897de405-6b4d-4333-81ff-d3add2f438cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:04:59.641955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 970/1909] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:04:59.645787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:59.787091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e80610-eda2-41c8-94eb-b0ec8d9e5b69 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:04:59.803064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:04:59.803971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:04:59.804315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:04:59.804530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:04:59.808280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:04:59.810383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:04:59.944155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 940/1910] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:04:59 2022] POST /identity/v3/projects => generated 401 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:04:59.946528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897de405-6b4d-4333-81ff-d3add2f438cc None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:04:59.957479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897de405-6b4d-4333-81ff-d3add2f438cc None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.040398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.041382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.041841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.042902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.047239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.047239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.188688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 971/1911] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:04:59 2022] POST /identity/v3/users => generated 307 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:00.192760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:00.204550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07677c6-f06f-46d6-90e6-721175dc15d2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.209182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d0bdc-a230-4744-a311-8249f4bf4911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.209793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d0bdc-a230-4744-a311-8249f4bf4911 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.210072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d0bdc-a230-4744-a311-8249f4bf4911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.210380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d0bdc-a230-4744-a311-8249f4bf4911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.211578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d0bdc-a230-4744-a311-8249f4bf4911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.213193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d0bdc-a230-4744-a311-8249f4bf4911 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.224855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 941/1912] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:00.229887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:00.240582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899e61f-56ae-4177-86fc-75927b6a7a5d admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.245419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.246335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.246453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.246696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] PATH_INFO: `/v3/projects/6ffecd8d1b8a45e682cf10744aa7ee5b/users/48f199d9aff24e12b2e3aec4bead8a72/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.248392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844dd70-ebe2-4db8-b683-d7a9ab3b5fdd None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.248766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844dd70-ebe2-4db8-b683-d7a9ab3b5fdd None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.249265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844dd70-ebe2-4db8-b683-d7a9ab3b5fdd None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.251031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 942/1913] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:05:00.261848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffecd8d1b8a45e682cf10744aa7ee5b, user_id=48f199d9aff24e12b2e3aec4bead8a72, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.264419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.264774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.264878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.265150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.287427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:00.289917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:00.292409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:00.360904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 972/1914] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:00 2022] PUT /identity/v3/projects/6ffecd8d1b8a45e682cf10744aa7ee5b/users/48f199d9aff24e12b2e3aec4bead8a72/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:00.362332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 943/1915] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:00.364978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:00.387990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:00.412218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:00.413678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:00.438270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cc6dc-b914-4c15-a11e-71f566ab3957 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.441758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b892-e16a-41c7-ac99-8edb36b1aa14 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.449391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.450159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.450573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.451344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.452128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.454008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.468488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 973/1916] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:00.472755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:00.518656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06cd6b-3324-4adc-929a-e57b7352d445 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.524699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.525373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.525652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.525980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] PATH_INFO: `/v3/projects/6ffecd8d1b8a45e682cf10744aa7ee5b/users/48f199d9aff24e12b2e3aec4bead8a72/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.528178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffecd8d1b8a45e682cf10744aa7ee5b, user_id=48f199d9aff24e12b2e3aec4bead8a72, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.529351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3007952-6eb4-4e58-9be3-41b295b4d7d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.529983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3007952-6eb4-4e58-9be3-41b295b4d7d6 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.530308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3007952-6eb4-4e58-9be3-41b295b4d7d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.530589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3007952-6eb4-4e58-9be3-41b295b4d7d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.530780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.541796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3007952-6eb4-4e58-9be3-41b295b4d7d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.543715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3007952-6eb4-4e58-9be3-41b295b4d7d6 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.544982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 974/1917] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:00 2022] PUT /identity/v3/projects/6ffecd8d1b8a45e682cf10744aa7ee5b/users/48f199d9aff24e12b2e3aec4bead8a72/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:00.547973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:00.556522 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:00.565274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 944/1918] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:00.566616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.567137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.567417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.603133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] MFA Rules not processed for user `48f199d9aff24e12b2e3aec4bead8a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:00.604356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:00.719566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77dc3c-b5c1-46a5-b684-3ca1f54e666c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.721098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 945/1919] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:00 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:00.734273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:00.735972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:00.744654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.745322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.745607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.745885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] PATH_INFO: `/v3/users/e216685fa53143a9b90563a0a44908bb` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.751131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e216685fa53143a9b90563a0a44908bb)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.754197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.855186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2ec78-da22-45f3-9c1c-8faa706ed9b2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:00.860924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:00.861571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:00.861869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:00.862132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:00.881021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:00.881021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:00.913234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 946/1920] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:00 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:00.976310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e216685fa53143a9b90563a0a44908bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:00.977079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e216685fa53143a9b90563a0a44908bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:00.977624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e216685fa53143a9b90563a0a44908bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:00.984554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e216685fa53143a9b90563a0a44908bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:01.131917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-da2605d3-28e4-4ab4-9520-2bbdfada5a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e216685fa53143a9b90563a0a44908bb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:01.158660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:01.160176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 975/1921] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:00 2022] DELETE /identity/v3/users/e216685fa53143a9b90563a0a44908bb => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:01.165386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:01.165888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:01.166357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:01.171667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aed83-01d8-4eb8-bd59-e2207d52e231 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:01.209591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:01.211334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:01.272404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:01.273041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:01.273498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:01.273797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:01.276369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:01.328492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 976/1922] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:01.358213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:01.359933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:01.380887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:01.382494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:01.388814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:01.409499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 947/1923] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:01.458883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd80c0-8451-4008-aebc-a9394ff00dc0 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:01.463846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:01.464599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:01.464944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:01.465289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:01.478565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:01.480311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:01.491276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 977/1924] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:01.543405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:02.206356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:02.206356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:02.231660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e45b39-fe52-4f89-b009-d4aadc301eb2 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:02.244667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:02.244667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:02.244667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:02.244667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] PATH_INFO: `/v3/projects/5674a257263b4fa7822291d25ca2ebeb` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:02.248714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5674a257263b4fa7822291d25ca2ebeb)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:02.251216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:02.301404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:02.400805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5674a257263b4fa7822291d25ca2ebeb', 'request_id': 'req-27c2fb0b-081a-4da7-83e6-496719f834d1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.411975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5674a257263b4fa7822291d25ca2ebeb', 'request_id': 'req-27c2fb0b-081a-4da7-83e6-496719f834d1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.420527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5674a257263b4fa7822291d25ca2ebeb', 'request_id': 'req-27c2fb0b-081a-4da7-83e6-496719f834d1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.516077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] The token cache is being invalidate because project 5674a257263b4fa7822291d25ca2ebeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:02.516584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.517580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.523150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 948/1925] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:02 2022] DELETE /identity/v3/projects/5674a257263b4fa7822291d25ca2ebeb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:02.524933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:02.538506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2cf4b-c665-4cef-baa5-f5a42d17ba62 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:02.571281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:02.655871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:02.676332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:02.676332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:02.676332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] PATH_INFO: `/v3/users/4cb0fec9808f4824b6a834345463007a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:02.676332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb0fec9808f4824b6a834345463007a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:02.676332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:02.908036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb0fec9808f4824b6a834345463007a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.913730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb0fec9808f4824b6a834345463007a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.913730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb0fec9808f4824b6a834345463007a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:02.927048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb0fec9808f4824b6a834345463007a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:03.018927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:03.018927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-bfee8810-f2d5-402b-8b23-839f57b9e7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb0fec9808f4824b6a834345463007a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:03.048400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 978/1926] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:02 2022] DELETE /identity/v3/users/4cb0fec9808f4824b6a834345463007a => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:03.507212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:03.662922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:03.764624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:03.766073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:03.788497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2fb0b-081a-4da7-83e6-496719f834d1 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:03.866266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:03.867493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:03.867988 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:03.868335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] PATH_INFO: `/v3/projects/6207a96e23954950ad534810ea49fa65` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:03.869608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6207a96e23954950ad534810ea49fa65)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:03.871676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:04.005151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6207a96e23954950ad534810ea49fa65', 'request_id': 'req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.011121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6207a96e23954950ad534810ea49fa65', 'request_id': 'req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.017270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6207a96e23954950ad534810ea49fa65', 'request_id': 'req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.144743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] The token cache is being invalidate because project 6207a96e23954950ad534810ea49fa65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:04.145291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.146139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dc8dfdaf-41ea-4064-ac23-a8384ef61d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.147911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 949/1927] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:03 2022] DELETE /identity/v3/projects/6207a96e23954950ad534810ea49fa65 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:04.153282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a316ea-a388-403f-8045-a69c531fbbb0 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.153756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a316ea-a388-403f-8045-a69c531fbbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.154182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a316ea-a388-403f-8045-a69c531fbbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.204128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-a4a316ea-a388-403f-8045-a69c531fbbb0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:04.205370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a4a316ea-a388-403f-8045-a69c531fbbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:04.323013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 979/1928] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:04.330353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358df416-868b-4d71-b0e6-0de7ced0b6ff None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.330849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358df416-868b-4d71-b0e6-0de7ced0b6ff None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.331316 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358df416-868b-4d71-b0e6-0de7ced0b6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.354024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-358df416-868b-4d71-b0e6-0de7ced0b6ff None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:04.355509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-358df416-868b-4d71-b0e6-0de7ced0b6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:04.364230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 950/1929] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:04.371630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a316ea-a388-403f-8045-a69c531fbbb0 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:04.395669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a316ea-a388-403f-8045-a69c531fbbb0 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:04.396216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:04.396825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.397104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.397378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.398032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:04.398605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:04.400232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:04.421938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 980/1930] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:04.426452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358df416-868b-4d71-b0e6-0de7ced0b6ff None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:04.442561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358df416-868b-4d71-b0e6-0de7ced0b6ff None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:04.457490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:04.458190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.458589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.458929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.459825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:04.461954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:04.598413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 951/1931] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:04 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:04.604227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:04.628044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:04.629735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:04.644919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e5705-bc39-463e-9460-b8aa3e7d47b6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:04.761185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d9c1f-6423-402e-b41f-aa32c0a84dc0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:04.761879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:04.762768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.763135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.763463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] PATH_INFO: `/v3/users/49ba9a1c10e3468c962379ffd58dd434` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.764827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ba9a1c10e3468c962379ffd58dd434)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:04.766984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:04.768190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:04.769491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.770099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.770575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.771355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:04.774384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:04.886791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 981/1932] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:04 2022] POST /identity/v3/users => generated 286 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:04.890553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:04.913273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c10ba-721e-41dc-90a3-3a67b9e2431a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:04.918826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:04.919446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.920303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.920707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.921825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:04.923495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:04.936642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 982/1933] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:04.941064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:04.951367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18605d-e346-4352-b08b-b63cbf2fb322 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:04.956807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:04.958127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:04.958544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:04.958884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:04.959612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:04.961483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:04.978420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 983/1934] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:04.981600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:04.990813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ba9a1c10e3468c962379ffd58dd434'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.992730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ba9a1c10e3468c962379ffd58dd434'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.993680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ba9a1c10e3468c962379ffd58dd434'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:04.995839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92fb5e-8d76-40bd-8981-d1a44a629691 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:05.001312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e021c28-f7e5-4592-8ede-75e9c3642f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:05.002049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021c28-f7e5-4592-8ede-75e9c3642f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:05.002486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021c28-f7e5-4592-8ede-75e9c3642f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:05.002770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021c28-f7e5-4592-8ede-75e9c3642f3e admin admin] PATH_INFO: `/v3/projects/832d2e1e764c403fbefd075a4a96d5b6/users/e7fa15fc3ec44a0c85f3c87327565280/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:05.020453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e021c28-f7e5-4592-8ede-75e9c3642f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=832d2e1e764c403fbefd075a4a96d5b6, user_id=e7fa15fc3ec44a0c85f3c87327565280, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:05.023175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e021c28-f7e5-4592-8ede-75e9c3642f3e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:05.072800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ba9a1c10e3468c962379ffd58dd434'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:05.080832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-74b64d82-d4e4-4458-9776-dd202ec511cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ba9a1c10e3468c962379ffd58dd434'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:05.099447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 952/1935] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:04 2022] DELETE /identity/v3/users/49ba9a1c10e3468c962379ffd58dd434 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:05.105131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:05.105131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:05.105131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:05.106181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 984/1936] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:04 2022] PUT /identity/v3/projects/832d2e1e764c403fbefd075a4a96d5b6/users/e7fa15fc3ec44a0c85f3c87327565280/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:05.111483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:05.111837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:05.112249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:05.125337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:05.126595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:05.133183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:05.134462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:05.193273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 953/1937] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:05.200499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 985/1938] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:05.213486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:05.217718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:05.219050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:05.233705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:05.298678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88272-da9c-4a09-9307-6bc117125f03 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:05.303274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:05.304091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:05.305081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:05.305476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:05.315162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:05.319202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:05.329333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 954/1939] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:05.330213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dc0a2-b42f-4247-9421-e9919b715eb8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:05.334779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:05.335701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:05.336129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:05.336543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:05.348320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:05.350567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:05.361350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 986/1940] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:05.959700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:06.016134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:06.207277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:06.208974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:06.232536 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae28f5f-208a-42da-bb81-cba72d0ec78f service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:06.246709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:06.247685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:06.248266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:06.248699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] PATH_INFO: `/v3/projects/af7e2d5bc411489d9a58e5638a2f854e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:06.250178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7e2d5bc411489d9a58e5638a2f854e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:06.253376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:06.398524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7e2d5bc411489d9a58e5638a2f854e', 'request_id': 'req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.404034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7e2d5bc411489d9a58e5638a2f854e', 'request_id': 'req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.412864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7e2d5bc411489d9a58e5638a2f854e', 'request_id': 'req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.501589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] The token cache is being invalidate because project af7e2d5bc411489d9a58e5638a2f854e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:06.501958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.502718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.504349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 955/1941] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:06 2022] DELETE /identity/v3/projects/af7e2d5bc411489d9a58e5638a2f854e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:06.508568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:06.518532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a445d-ca91-4c62-a097-69e037d0374c service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:06.575450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:06.576383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:06.576812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:06.577234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] PATH_INFO: `/v3/users/7544f00e764e4779af1fa6802450ea04` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:06.578676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7544f00e764e4779af1fa6802450ea04)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:06.582630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:06.828659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7544f00e764e4779af1fa6802450ea04'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.829889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7544f00e764e4779af1fa6802450ea04'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.830197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7544f00e764e4779af1fa6802450ea04'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.841575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7544f00e764e4779af1fa6802450ea04'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.906653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be9a4e60-7092-4f5b-930d-1075023e0779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7544f00e764e4779af1fa6802450ea04'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:06.925389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 987/1942] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:06 2022] DELETE /identity/v3/users/7544f00e764e4779af1fa6802450ea04 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:06.994645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:06.996116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:07.011039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6dc5d-53d8-4413-8c37-a4cecab5127c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:07.067309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:07.077071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:07.077449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:07.077633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:07.078073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] PATH_INFO: `/v3/users/bbccd9c7036c4685a33a6af82ba3e344` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:07.083063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbccd9c7036c4685a33a6af82ba3e344)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:07.085719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:07.124732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:07.207042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:07.207424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:07.207646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:07.253830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] MFA Rules not processed for user `e7fa15fc3ec44a0c85f3c87327565280`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:07.255982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:07.313044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbccd9c7036c4685a33a6af82ba3e344'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.313044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbccd9c7036c4685a33a6af82ba3e344'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.314754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbccd9c7036c4685a33a6af82ba3e344'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.367320 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 988/1943] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:07.372363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:07.374343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:07.392274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d3cc2-8367-4186-8a34-dbdae2670b10 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:07.399940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbccd9c7036c4685a33a6af82ba3e344'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.415115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-81f5493b-19f3-43c0-ab97-af07f5991263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbccd9c7036c4685a33a6af82ba3e344'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.423487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:07.426551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:07.426993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:07.427412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:07.432240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:07.434458 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:07.472178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 956/1944] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:06 2022] DELETE /identity/v3/users/bbccd9c7036c4685a33a6af82ba3e344 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:07.477819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e778c91-d3fe-4a31-884e-fc14e5cbf6b8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:07.478360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e778c91-d3fe-4a31-884e-fc14e5cbf6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:07.478644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e778c91-d3fe-4a31-884e-fc14e5cbf6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:07.501775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3e778c91-d3fe-4a31-884e-fc14e5cbf6b8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:07.503061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3e778c91-d3fe-4a31-884e-fc14e5cbf6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:07.544227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:07.556728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 989/1945] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:07 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:07.559828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:07.561582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:07.588245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 957/1946] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:07.589563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e778c91-d3fe-4a31-884e-fc14e5cbf6b8 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:07.602786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e778c91-d3fe-4a31-884e-fc14e5cbf6b8 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:07.661700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86372318-a84c-4ed7-af23-e7bb80256fc5 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:07.666679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:07.667299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:07.667583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:07.667861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] PATH_INFO: `/v3/projects/22010ba4f41f4b26ab0cf6471d5267a1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:07.669183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] RBAC: Authorizing `identity:delete_project(project_id=22010ba4f41f4b26ab0cf6471d5267a1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:07.673603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:07.691931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:07.692575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:07.692865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:07.693139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:07.696417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:07.698345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:07.823376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 958/1947] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:07 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:07.824330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:07.827747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:07.847171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22010ba4f41f4b26ab0cf6471d5267a1', 'request_id': 'req-b2b97738-6b9a-4f8c-b7f0-68096959225f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.867612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22010ba4f41f4b26ab0cf6471d5267a1', 'request_id': 'req-b2b97738-6b9a-4f8c-b7f0-68096959225f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.883912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22010ba4f41f4b26ab0cf6471d5267a1', 'request_id': 'req-b2b97738-6b9a-4f8c-b7f0-68096959225f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.962256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d79e-2a3e-4e0d-bbd5-7abeae89475d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:07.984237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] The token cache is being invalidate because project 22010ba4f41f4b26ab0cf6471d5267a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:07.984796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.985964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:07.991107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 990/1948] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:07 2022] DELETE /identity/v3/projects/22010ba4f41f4b26ab0cf6471d5267a1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:07.992635 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:07.994185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:08.061961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:08.064078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.064733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.065837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.066137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.075956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:08.084755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b97738-6b9a-4f8c-b7f0-68096959225f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.091752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7729c8-d2fa-40ea-bffa-6f877a760eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.092614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7729c8-d2fa-40ea-bffa-6f877a760eae admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.092900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7729c8-d2fa-40ea-bffa-6f877a760eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.100642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7729c8-d2fa-40ea-bffa-6f877a760eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.100767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.101283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7729c8-d2fa-40ea-bffa-6f877a760eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.101283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7729c8-d2fa-40ea-bffa-6f877a760eae admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.101479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.108865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 991/1949] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:08.110619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 959/1950] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:08.110740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469cba-f274-4345-85bb-1e167f6600e7 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.111231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469cba-f274-4345-85bb-1e167f6600e7 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.111526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0469cba-f274-4345-85bb-1e167f6600e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.119769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.135524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9735e-caed-4b8e-a3d5-1bbc614ee6d8 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.143993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.145460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.146000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.146370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] PATH_INFO: `/v3/projects/1dc2676bce2043c7adab9397677930d5/users/79a3479d79d64f349ed7356fb16f3193/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.157436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c0469cba-f274-4345-85bb-1e167f6600e7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:08.158762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c0469cba-f274-4345-85bb-1e167f6600e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:08.169343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 992/1951] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:08.169859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc2676bce2043c7adab9397677930d5, user_id=79a3479d79d64f349ed7356fb16f3193, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.173174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.179804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871996c5-e672-4c43-8a0a-3ae745024845 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.180165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871996c5-e672-4c43-8a0a-3ae745024845 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.180449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871996c5-e672-4c43-8a0a-3ae745024845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.203137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-871996c5-e672-4c43-8a0a-3ae745024845 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:08.204346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-871996c5-e672-4c43-8a0a-3ae745024845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:08.213906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 993/1952] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:08.218350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871996c5-e672-4c43-8a0a-3ae745024845 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.229416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871996c5-e672-4c43-8a0a-3ae745024845 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.234975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.235627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.236616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.236616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.236930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:08.237264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.239149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.252532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 994/1953] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:08.257106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.268184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 960/1954] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:08 2022] PUT /identity/v3/projects/1dc2676bce2043c7adab9397677930d5/users/79a3479d79d64f349ed7356fb16f3193/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:08.272732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.275169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02fb5-8088-4cab-a791-dfa81e07745b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.364540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.365125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.365426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.365735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.366432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.366585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f52ba2-6181-4981-b3bc-2e69c509740c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.371176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.372683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.373503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.373858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.374379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.375130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.376812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.395784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 961/1955] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:08.396891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.409681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a94d39-b1ce-4abe-9553-1cd18c1fd3db admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.417868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.418884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.419462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.425955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] PATH_INFO: `/v3/projects/1dc2676bce2043c7adab9397677930d5/users/79a3479d79d64f349ed7356fb16f3193/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.425955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc2676bce2043c7adab9397677930d5, user_id=79a3479d79d64f349ed7356fb16f3193, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.425955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.442857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 962/1956] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:08 2022] PUT /identity/v3/projects/1dc2676bce2043c7adab9397677930d5/users/79a3479d79d64f349ed7356fb16f3193/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:08.445506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:08.490669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 995/1957] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:08 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:08.494565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.575262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93f9c-b2b6-4be2-90bb-c310066842b1 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.641498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.642741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.643240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.643708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.644555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.647750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.770531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 963/1958] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:08 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:08.773214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.786265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc51cc-757d-44b4-b151-b4a52145508a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.795489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.795489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.795489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.795489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.795798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.796334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.822632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 996/1959] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:08.826834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.840131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a33f0-a029-471c-8772-ea38f65d227b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.844704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.845317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.845593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.845865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.846542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.848230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.879333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 964/1960] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:08.881082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.895509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52161-a86f-4bb2-b1e7-0b8437fafd4b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.901820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.902730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.903116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.903497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] PATH_INFO: `/v3/projects/f7df3cf6580c422089bfaf83ee235c9f/users/566ee7dc1fa641f38bb041fa115d2a40/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.921909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:08.922081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7df3cf6580c422089bfaf83ee235c9f, user_id=566ee7dc1fa641f38bb041fa115d2a40, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.923724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:08.925045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:08.941431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe555a-03de-462b-86df-c8574fa04cd9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:08.947139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:08.947835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:08.948121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:08.953265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] PATH_INFO: `/v3/projects/c421eec5f3984c309286c7cf3a23700d` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:08.953265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c421eec5f3984c309286c7cf3a23700d)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:08.953265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:09.010654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 997/1961] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:08 2022] PUT /identity/v3/projects/f7df3cf6580c422089bfaf83ee235c9f/users/566ee7dc1fa641f38bb041fa115d2a40/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:09.011625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:09.020887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:09.119298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c421eec5f3984c309286c7cf3a23700d', 'request_id': 'req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.123046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde4a9d-507f-446f-a408-f91676f13486 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:09.129338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c421eec5f3984c309286c7cf3a23700d', 'request_id': 'req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.140847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c421eec5f3984c309286c7cf3a23700d', 'request_id': 'req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.200231 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:09.200801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:09.201215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:09.201535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] PATH_INFO: `/v3/users/e44727cbe0d9471cbf788127d82b606b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:09.202807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e44727cbe0d9471cbf788127d82b606b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:09.204730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:09.227219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] The token cache is being invalidate because project c421eec5f3984c309286c7cf3a23700d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:09.227219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.227596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5b043689-8e3e-45aa-b3b5-8b9fb70435d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.232272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 965/1962] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:08 2022] DELETE /identity/v3/projects/c421eec5f3984c309286c7cf3a23700d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:09.232272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330111d-f931-4771-87d0-982b21457a65 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:09.232272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330111d-f931-4771-87d0-982b21457a65 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:09.232272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330111d-f931-4771-87d0-982b21457a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:09.320907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0330111d-f931-4771-87d0-982b21457a65 None None] MFA Rules not processed for user `566ee7dc1fa641f38bb041fa115d2a40`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:09.322258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0330111d-f931-4771-87d0-982b21457a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:09.431311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e44727cbe0d9471cbf788127d82b606b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.432107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e44727cbe0d9471cbf788127d82b606b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.432691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e44727cbe0d9471cbf788127d82b606b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.444914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e44727cbe0d9471cbf788127d82b606b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.473052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 966/1963] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:09.474187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330111d-f931-4771-87d0-982b21457a65 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:09.474633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:09.475739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0330111d-f931-4771-87d0-982b21457a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:09.493486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330111d-f931-4771-87d0-982b21457a65 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:09.499137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:09.501244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:09.501244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:09.501244 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] PATH_INFO: `/v3/users/48f199d9aff24e12b2e3aec4bead8a72` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:09.501807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f199d9aff24e12b2e3aec4bead8a72)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:09.504119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:09.512671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e44727cbe0d9471cbf788127d82b606b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.546853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 998/1964] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:09 2022] DELETE /identity/v3/users/e44727cbe0d9471cbf788127d82b606b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:09.547518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:09.549490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:09.636017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47784a-1943-4695-8ec9-b5a8bba09f5d admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:09.643026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c63ef-8387-48e4-a304-4496e5185259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:09.643823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c63ef-8387-48e4-a304-4496e5185259 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:09.644116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c63ef-8387-48e4-a304-4496e5185259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:09.644973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c63ef-8387-48e4-a304-4496e5185259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:09.645084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c63ef-8387-48e4-a304-4496e5185259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:09.647269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c63ef-8387-48e4-a304-4496e5185259 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:09.723154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f199d9aff24e12b2e3aec4bead8a72'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.723823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f199d9aff24e12b2e3aec4bead8a72'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.724434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f199d9aff24e12b2e3aec4bead8a72'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.767194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 999/1965] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:09 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:09.768920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:09.769213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:09.769491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:09.789810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:09.791062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:09.809337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f199d9aff24e12b2e3aec4bead8a72'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.818995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f199d9aff24e12b2e3aec4bead8a72'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:09.849412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 967/1966] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:09 2022] DELETE /identity/v3/users/48f199d9aff24e12b2e3aec4bead8a72 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:09.852501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:09.867064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1000/1967] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:09.875049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:09.885325 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:09.887009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:09.958177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df331f56-3971-4799-b3e5-e535617125b3 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:09.966222 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:09.967017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:09.967377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:09.967710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:09.968689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:09.970785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:09.996420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06115c3c-ed4d-4f45-8bdb-51ed6ec632e8 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:10.052843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:10.056017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.056017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.056017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.078830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:10.080774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:10.097209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 968/1968] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:09 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:10.098496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:10.112167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1001/1969] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:10.115889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceec01f-0a21-4e7c-ad9a-2e82dd83feae admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:10.122407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:10.123046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.123377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.123839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.124499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:10.130593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:10.141887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 969/1970] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:10.143104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:10.170252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4905389-8cde-4041-83c9-86ca9e896850 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:10.193585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:10.194477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.195442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.196077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] PATH_INFO: `/v3/projects/f5a9b29c89d74b6e98302dc432f9196a/users/5535bf4da8ad4149997e988189492cf1/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.228174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a9b29c89d74b6e98302dc432f9196a, user_id=5535bf4da8ad4149997e988189492cf1, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:10.231092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:10.321119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1002/1971] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:10 2022] PUT /identity/v3/projects/f5a9b29c89d74b6e98302dc432f9196a/users/5535bf4da8ad4149997e988189492cf1/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:10.325622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:10.345143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e046ac-cfe7-4590-8ce4-dd9e0959505a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:10.443973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:10.471163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:10.480529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.480925 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.481207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.488075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:10.489910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:10.494709 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:10.494885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:10.519694 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 970/1972] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:10.525654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:10.547991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e9e0-72cc-4a94-9803-dbbd8a6bf9bb admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:10.560564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07c1f8-10ef-4755-9c4c-5b1233d5aa70 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:10.568949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721f57a-cd9c-4ffb-8e04-61267944e736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:10.571506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f57a-cd9c-4ffb-8e04-61267944e736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.572513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f57a-cd9c-4ffb-8e04-61267944e736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.573401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f57a-cd9c-4ffb-8e04-61267944e736 admin admin] PATH_INFO: `/v3/projects/f5a9b29c89d74b6e98302dc432f9196a/users/5535bf4da8ad4149997e988189492cf1/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.576498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721f57a-cd9c-4ffb-8e04-61267944e736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a9b29c89d74b6e98302dc432f9196a, user_id=5535bf4da8ad4149997e988189492cf1, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:10.577415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:10.579758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.580029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721f57a-cd9c-4ffb-8e04-61267944e736 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:10.589423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.589423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] PATH_INFO: `/v3/projects/6ffecd8d1b8a45e682cf10744aa7ee5b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.595519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffecd8d1b8a45e682cf10744aa7ee5b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:10.601703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 971/1973] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:10 2022] PUT /identity/v3/projects/f5a9b29c89d74b6e98302dc432f9196a/users/5535bf4da8ad4149997e988189492cf1/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:10.602061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:10.611977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83696a9c-c54a-4d84-8dcc-13be99dc01bf None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.612265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83696a9c-c54a-4d84-8dcc-13be99dc01bf None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.612547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83696a9c-c54a-4d84-8dcc-13be99dc01bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.691697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-83696a9c-c54a-4d84-8dcc-13be99dc01bf None None] MFA Rules not processed for user `5535bf4da8ad4149997e988189492cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:10.693174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-83696a9c-c54a-4d84-8dcc-13be99dc01bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:10.786702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffecd8d1b8a45e682cf10744aa7ee5b', 'request_id': 'req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:10.807213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffecd8d1b8a45e682cf10744aa7ee5b', 'request_id': 'req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:10.832889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffecd8d1b8a45e682cf10744aa7ee5b', 'request_id': 'req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:10.870496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 972/1974] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:10.871969 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ceaed-00fc-4172-b190-43f532d934cc None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:10.872264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ceaed-00fc-4172-b190-43f532d934cc None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:10.872545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ceaed-00fc-4172-b190-43f532d934cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:10.906246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c92ceaed-00fc-4172-b190-43f532d934cc None None] MFA Rules not processed for user `79a3479d79d64f349ed7356fb16f3193`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:10.907640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c92ceaed-00fc-4172-b190-43f532d934cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:10.930564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] The token cache is being invalidate because project 6ffecd8d1b8a45e682cf10744aa7ee5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:10.930855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:10.931550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:10.933395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1003/1975] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:10 2022] DELETE /identity/v3/projects/6ffecd8d1b8a45e682cf10744aa7ee5b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:10.934200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:10.935724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.034776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 973/1976] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:11.037147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.037147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.037147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.064369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:11.065891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.072556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcc7ad-28e2-408a-a1a4-bd11b35108a6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.085457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270bd31-9893-4d82-8450-ebff18bf8728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.086094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270bd31-9893-4d82-8450-ebff18bf8728 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.086402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270bd31-9893-4d82-8450-ebff18bf8728 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.086850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270bd31-9893-4d82-8450-ebff18bf8728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.091935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7270bd31-9893-4d82-8450-ebff18bf8728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.165688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 974/1977] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:11.166638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.168476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.191932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dccca6-ec8c-4bfc-86b8-c04a3ba4fa25 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.197110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.197963 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.198272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.198566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.200363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.206550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270bd31-9893-4d82-8450-ebff18bf8728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.209847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7270bd31-9893-4d82-8450-ebff18bf8728 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.223748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1004/1978] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:11.225483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc47a7c-9d18-4cf7-9b4a-9c865439e261 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.226453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc47a7c-9d18-4cf7-9b4a-9c865439e261 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.226827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc47a7c-9d18-4cf7-9b4a-9c865439e261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.264459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9dc47a7c-9d18-4cf7-9b4a-9c865439e261 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:11.266042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9dc47a7c-9d18-4cf7-9b4a-9c865439e261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.275417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1005/1979] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:11.279795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc47a7c-9d18-4cf7-9b4a-9c865439e261 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.284434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.286039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.294364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc47a7c-9d18-4cf7-9b4a-9c865439e261 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.297684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 975/1980] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:11.301758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.302821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.303007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.303279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.304628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:11.304935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.306785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.319595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1006/1981] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:11.324580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.335974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdafaecc-c20f-4456-8d55-130108b3966d service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.342880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.343504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.343791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.344068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.344696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.346663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.382015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.387811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.407750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80357e08-8d8e-4597-b9e5-7abd97b0b168 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.412889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.414180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.414180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.414180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] PATH_INFO: `/v3/projects/f677f169706a426d8e3efd823f29dbdb` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.415665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f677f169706a426d8e3efd823f29dbdb)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.417971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.450738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 976/1982] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:11.451834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.454636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:11.526918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3b31d-a901-417b-934f-649a7d5f4aba admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.534905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f677f169706a426d8e3efd823f29dbdb', 'request_id': 'req-586240a0-3f04-4070-9c05-5d43abd94894'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:11.541005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f677f169706a426d8e3efd823f29dbdb', 'request_id': 'req-586240a0-3f04-4070-9c05-5d43abd94894'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:11.547946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f677f169706a426d8e3efd823f29dbdb', 'request_id': 'req-586240a0-3f04-4070-9c05-5d43abd94894'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:11.588806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.589694 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.589880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.590403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.590942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.594627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.621010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] The token cache is being invalidate because project f677f169706a426d8e3efd823f29dbdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:11.621334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:11.621986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-586240a0-3f04-4070-9c05-5d43abd94894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:11.623521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1007/1983] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:11 2022] DELETE /identity/v3/projects/f677f169706a426d8e3efd823f29dbdb => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:11.628198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba8ae-7969-426f-bf03-6dcdc721d36f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.628485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba8ae-7969-426f-bf03-6dcdc721d36f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.628783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aba8ae-7969-426f-bf03-6dcdc721d36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.648596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-85aba8ae-7969-426f-bf03-6dcdc721d36f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:11.649860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-85aba8ae-7969-426f-bf03-6dcdc721d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.658020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1008/1984] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:11.666297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd8f1-c276-4463-b1cd-011613b83833 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.666591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd8f1-c276-4463-b1cd-011613b83833 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.666866 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dd8f1-c276-4463-b1cd-011613b83833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.686977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-af0dd8f1-c276-4463-b1cd-011613b83833 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:11.688516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-af0dd8f1-c276-4463-b1cd-011613b83833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.690487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 977/1985] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/users => generated 291 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:11.694727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.697187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.703176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1009/1986] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:11.707279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dd8f1-c276-4463-b1cd-011613b83833 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.712568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fff10c-093c-4068-93bc-eacfd5b36446 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.718797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dd8f1-c276-4463-b1cd-011613b83833 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.719190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.721505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.721789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.722064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.723512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.724853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.732737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.733367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.733676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.733960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.734936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:11.735243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.736838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.744913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 978/1987] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:11.750940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.752920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1010/1988] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:05:11.753777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.761981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a52a-f8c4-4b76-8a11-6164fd92c1b8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.764607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7834b-9e0c-440c-8186-8578bd44183f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.769121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.769731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.769846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.770008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.770309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.770453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.770748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.770956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.771073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.771640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.773067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.774345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.784790 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 979/1989] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:11.789739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.801300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec82ad-6d2a-4979-9ed3-f1eca066a19a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:11.806885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f6352-ff3a-46d4-a53b-c4434f218d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:11.807956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6352-ff3a-46d4-a53b-c4434f218d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.808679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6352-ff3a-46d4-a53b-c4434f218d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.809179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f6352-ff3a-46d4-a53b-c4434f218d09 admin admin] PATH_INFO: `/v3/projects/adb6956033134950b8f647a20ef77f56/users/bf8c785fd79949d0bc49485deab8bdb9/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.829091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f6352-ff3a-46d4-a53b-c4434f218d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adb6956033134950b8f647a20ef77f56, user_id=bf8c785fd79949d0bc49485deab8bdb9, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:11.830688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f6352-ff3a-46d4-a53b-c4434f218d09 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:11.890939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1011/1990] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/projects => generated 397 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:11.903555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:11.919954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 980/1991] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:11 2022] PUT /identity/v3/projects/adb6956033134950b8f647a20ef77f56/users/bf8c785fd79949d0bc49485deab8bdb9/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:11.927878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48866-95ab-4b96-a549-21051aaf494a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:11.928159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48866-95ab-4b96-a549-21051aaf494a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:11.928432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48866-95ab-4b96-a549-21051aaf494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:11.977549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e1c48866-95ab-4b96-a549-21051aaf494a None None] MFA Rules not processed for user `bf8c785fd79949d0bc49485deab8bdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:11.978893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e1c48866-95ab-4b96-a549-21051aaf494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:11.993281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:12.014122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941f996-75d2-4ff5-9274-bba196e4ebcc admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.076396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.077009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.077409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.077692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.078351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.080243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.080797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 981/1992] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:12.085767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48866-95ab-4b96-a549-21051aaf494a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.096433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48866-95ab-4b96-a549-21051aaf494a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.104800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.105416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.105699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.105974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.106644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.108567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.225084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1012/1993] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:11 2022] POST /identity/v3/users => generated 299 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:12.236812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.248666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:12.257479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e339e-a97d-473c-a148-4e96bb25fcd2 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.264683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.265265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.265653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.265938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.266638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.268287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.270855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 982/1994] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:12 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:12.272201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.283528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1013/1995] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:12.286299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.305615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98035f3c-2561-4ff9-90f4-851499d0bd3d admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.387663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c5544-2680-4bd5-97e4-0ac6500fb679 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.389318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.390120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.390440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.391445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.392170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.394521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.398811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.399441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.399724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.399997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.400630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.409933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.414836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1014/1996] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:12.417915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.431524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae81a8-50d7-4f5f-8d56-9d024537ad93 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.437367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa17e73-0715-4c60-864a-9de26691ad7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.438030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa17e73-0715-4c60-864a-9de26691ad7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.438347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa17e73-0715-4c60-864a-9de26691ad7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.438639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa17e73-0715-4c60-864a-9de26691ad7c admin admin] PATH_INFO: `/v3/projects/d09edb47b6c64c89b148e65de5bfeee2/users/6914b983117c4f0aa3a0365e0798d71e/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.456236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa17e73-0715-4c60-864a-9de26691ad7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09edb47b6c64c89b148e65de5bfeee2, user_id=6914b983117c4f0aa3a0365e0798d71e, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.459021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa17e73-0715-4c60-864a-9de26691ad7c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.508234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 983/1997] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:12 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:12.515350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.534567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1015/1998] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:12 2022] PUT /identity/v3/projects/d09edb47b6c64c89b148e65de5bfeee2/users/6914b983117c4f0aa3a0365e0798d71e/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:12.537462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b068c-d950-480d-8560-48bbfe128fa0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.547553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dddcf5-b4ba-4873-9837-4ed1d4b62f16 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.547855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dddcf5-b4ba-4873-9837-4ed1d4b62f16 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.548146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dddcf5-b4ba-4873-9837-4ed1d4b62f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.589945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-a6dddcf5-b4ba-4873-9837-4ed1d4b62f16 None None] MFA Rules not processed for user `6914b983117c4f0aa3a0365e0798d71e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:12.591287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a6dddcf5-b4ba-4873-9837-4ed1d4b62f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:12.614549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.615172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.615490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.615719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.617667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.618203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.634814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 984/1999] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:12.642045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.654798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d4558-d2f1-4108-85e9-282d7c78195c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.660998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.661886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.662686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.663677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] PATH_INFO: `/v3/projects/c73d813f16624234af0de2f8e8abc5a0/users/894539db640e45aeaa0a0ec9342f64ba/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.682789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c73d813f16624234af0de2f8e8abc5a0, user_id=894539db640e45aeaa0a0ec9342f64ba, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.683941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1016/2000] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:12.692802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.832228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 985/2001] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:12 2022] PUT /identity/v3/projects/c73d813f16624234af0de2f8e8abc5a0/users/894539db640e45aeaa0a0ec9342f64ba/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:12.837255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dddcf5-b4ba-4873-9837-4ed1d4b62f16 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.841228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.842912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:12.848970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dddcf5-b4ba-4873-9837-4ed1d4b62f16 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.903208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9bbc0-4857-411e-89c8-14f0d8afa805 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.907406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.908025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.908312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.908581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.909590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.910371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.910653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.910930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.911576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.913871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.923939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1017/2002] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:12.928260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:12.939570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bf228-2e1c-432e-a807-757e5c09f771 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:12.944683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec4b43-18f1-4958-b2ff-b1047ff30fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:12.945342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4b43-18f1-4958-b2ff-b1047ff30fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.946375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4b43-18f1-4958-b2ff-b1047ff30fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.946375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec4b43-18f1-4958-b2ff-b1047ff30fb1 admin admin] PATH_INFO: `/v3/projects/c73d813f16624234af0de2f8e8abc5a0/users/894539db640e45aeaa0a0ec9342f64ba/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.948038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec4b43-18f1-4958-b2ff-b1047ff30fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c73d813f16624234af0de2f8e8abc5a0, user_id=894539db640e45aeaa0a0ec9342f64ba, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.950930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec4b43-18f1-4958-b2ff-b1047ff30fb1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.965010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1018/2003] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:12 2022] PUT /identity/v3/projects/c73d813f16624234af0de2f8e8abc5a0/users/894539db640e45aeaa0a0ec9342f64ba/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:12.976886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47a56d-cf98-4fe4-a789-545f4682dffc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:12.978004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47a56d-cf98-4fe4-a789-545f4682dffc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:12.978373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:12.978977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47a56d-cf98-4fe4-a789-545f4682dffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:12.979882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:12.992004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 986/2004] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:13.018183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bd47a56d-cf98-4fe4-a789-545f4682dffc None None] MFA Rules not processed for user `894539db640e45aeaa0a0ec9342f64ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:13.021597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bd47a56d-cf98-4fe4-a789-545f4682dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:13.130290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1019/2005] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:13.265866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:13.267302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:13.278653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:13.281581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b67d1-848f-439a-b2e9-bb35c24cbfdf service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:13.344483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:13.353401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:13.354111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:13.354351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:13.354804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:13.365682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:13.367245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:13.377229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 987/2006] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:13.415374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47a56d-cf98-4fe4-a789-545f4682dffc None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:13.424371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:13.430785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:13.431234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47a56d-cf98-4fe4-a789-545f4682dffc None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:13.439437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1509675-d940-4690-ab19-fcc5c4f52c25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:13.440069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1509675-d940-4690-ab19-fcc5c4f52c25 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:13.440348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1509675-d940-4690-ab19-fcc5c4f52c25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:13.440715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1509675-d940-4690-ab19-fcc5c4f52c25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:13.442497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e1509675-d940-4690-ab19-fcc5c4f52c25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:13.513725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:13.547371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1509675-d940-4690-ab19-fcc5c4f52c25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:13.550732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1509675-d940-4690-ab19-fcc5c4f52c25 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:13.550849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b422ae8-da4c-4c2b-9409-7733058bd04b service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:13.567915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1020/2007] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:13.578447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:13.579252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:13.579533 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:13.579812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] PATH_INFO: `/v3/users/566ee7dc1fa641f38bb041fa115d2a40` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:13.583313 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=566ee7dc1fa641f38bb041fa115d2a40)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:13.589957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:13.840785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:13.886994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566ee7dc1fa641f38bb041fa115d2a40'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:13.887876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566ee7dc1fa641f38bb041fa115d2a40'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:13.888766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566ee7dc1fa641f38bb041fa115d2a40'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:13.960996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566ee7dc1fa641f38bb041fa115d2a40'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:13.974741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566ee7dc1fa641f38bb041fa115d2a40'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:14.000665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 988/2008] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:13 2022] DELETE /identity/v3/users/566ee7dc1fa641f38bb041fa115d2a40 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:14.002207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:14.003024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:14.003024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:14.009588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:14.026765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:14.028340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:14.056045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:14.058509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:14.125769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1021/2009] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:14.136915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:14.138680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:14.142569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc59d7d-56ba-4a38-84b2-80e3ab034b5b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:14.148195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:14.148970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:14.149256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:14.149532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:14.151811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:14.221423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c2c46-bcec-454c-8a50-c2ede885a5fb None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:14.226170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:14.227574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:14.227819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:14.228092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:14.242787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:14.251549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:14.257515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1022/2010] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:14.257868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:14.259652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:14.269431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 989/2011] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:14.326342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:14.644070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:14.799284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:14.800822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:14.815592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93693cd-0a30-4894-ad24-ba1d5432815b service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:14.820824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:14.821462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:14.821742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:14.822021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] PATH_INFO: `/v3/projects/f7df3cf6580c422089bfaf83ee235c9f` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:14.823372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7df3cf6580c422089bfaf83ee235c9f)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:14.825223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:14.969194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7df3cf6580c422089bfaf83ee235c9f', 'request_id': 'req-9ab253d2-739f-446d-84d5-d73fe7c549e6'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:14.974489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7df3cf6580c422089bfaf83ee235c9f', 'request_id': 'req-9ab253d2-739f-446d-84d5-d73fe7c549e6'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:14.981220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7df3cf6580c422089bfaf83ee235c9f', 'request_id': 'req-9ab253d2-739f-446d-84d5-d73fe7c549e6'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.124971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] The token cache is being invalidate because project f7df3cf6580c422089bfaf83ee235c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:15.125180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.125795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.127331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1023/2012] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:14 2022] DELETE /identity/v3/projects/f7df3cf6580c422089bfaf83ee235c9f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:15.131204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:15.142565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069953b-4a28-4fcf-9861-cda1ee7efd63 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:15.200082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:15.200721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:15.201003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:15.201278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] PATH_INFO: `/v3/users/5535bf4da8ad4149997e988189492cf1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:15.202411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=5535bf4da8ad4149997e988189492cf1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:15.204321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:15.260696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:15.357947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:15.401767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5535bf4da8ad4149997e988189492cf1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.402940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5535bf4da8ad4149997e988189492cf1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.403230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5535bf4da8ad4149997e988189492cf1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.478341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5535bf4da8ad4149997e988189492cf1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.489116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9c98689d-9f67-4b2d-873c-63e0c5d21dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5535bf4da8ad4149997e988189492cf1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:15.510710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 990/2013] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:15 2022] DELETE /identity/v3/users/5535bf4da8ad4149997e988189492cf1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:15.902151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:15.953857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:15.964395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:15.981624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab253d2-739f-446d-84d5-d73fe7c549e6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:16.050157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:16.073545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:16.074195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.074533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.074808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] PATH_INFO: `/v3/projects/f5a9b29c89d74b6e98302dc432f9196a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.075939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a9b29c89d74b6e98302dc432f9196a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:16.077893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:16.185634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a9b29c89d74b6e98302dc432f9196a', 'request_id': 'req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:16.191094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a9b29c89d74b6e98302dc432f9196a', 'request_id': 'req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:16.198116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a9b29c89d74b6e98302dc432f9196a', 'request_id': 'req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:16.265587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] The token cache is being invalidate because project f5a9b29c89d74b6e98302dc432f9196a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:16.265878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:16.266525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4ede34fa-b82f-4b3c-a146-fa04c9cbd856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:16.268030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1024/2014] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:15 2022] DELETE /identity/v3/projects/f5a9b29c89d74b6e98302dc432f9196a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:16.272858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a20298-c1e4-47cf-87ea-064f250d15d5 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.273149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a20298-c1e4-47cf-87ea-064f250d15d5 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.273425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a20298-c1e4-47cf-87ea-064f250d15d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.293398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e1a20298-c1e4-47cf-87ea-064f250d15d5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:16.294650 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e1a20298-c1e4-47cf-87ea-064f250d15d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:16.352336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 991/2015] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:16.361419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1bc9-d1f5-4218-b831-fc9aed6a82de None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.361708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1bc9-d1f5-4218-b831-fc9aed6a82de None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.361976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb1bc9-d1f5-4218-b831-fc9aed6a82de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.381285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4deb1bc9-d1f5-4218-b831-fc9aed6a82de None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:16.382517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4deb1bc9-d1f5-4218-b831-fc9aed6a82de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:16.390720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1025/2016] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:16.394965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a20298-c1e4-47cf-87ea-064f250d15d5 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:16.405828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a20298-c1e4-47cf-87ea-064f250d15d5 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:16.410494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:16.416104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.416104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.416104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.416104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:16.416104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:16.416104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:16.425974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 992/2017] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:05:16.430272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1bc9-d1f5-4218-b831-fc9aed6a82de None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:16.441436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb1bc9-d1f5-4218-b831-fc9aed6a82de None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:16.447329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:16.448145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.448604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.449269 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.451515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:16.452148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:16.524372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1026/2018] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:16 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:16.524576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:16.527775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:16.537442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921fef2-9156-41c3-9554-f73947b1a371 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:16.592107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:16.592710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.592987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.593258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.593865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:16.595879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:16.677631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 993/2019] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:16 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:16.681570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:16.691748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dace8-4d12-4972-a50e-af2d1cd64b5b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:16.696742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:16.697360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.697640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.697912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.698636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:16.700219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:16.712939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1027/2020] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:16.714758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:16.724818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c955c1d-5990-4c94-90c1-21ee1235da1d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:16.730792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421dd0e-964e-45af-873b-5f537ff6bc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:16.731404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421dd0e-964e-45af-873b-5f537ff6bc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.731687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421dd0e-964e-45af-873b-5f537ff6bc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.731961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421dd0e-964e-45af-873b-5f537ff6bc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.732605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421dd0e-964e-45af-873b-5f537ff6bc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:16.734554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421dd0e-964e-45af-873b-5f537ff6bc4b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:16.745344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 994/2021] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:16.749470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:16.759784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06666443-1070-4180-a70c-f1c82aa6be4a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:16.764328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:16.764966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.765240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.765513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] PATH_INFO: `/v3/projects/47ec6712d1504ee58040e3bf7e904993/users/70046c1f26334c8a960acd8f301549c5/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.780933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ec6712d1504ee58040e3bf7e904993, user_id=70046c1f26334c8a960acd8f301549c5, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:16.783612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:16.826633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:16.884717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1028/2022] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:16 2022] PUT /identity/v3/projects/47ec6712d1504ee58040e3bf7e904993/users/70046c1f26334c8a960acd8f301549c5/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:16.895239 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457ea9d-8566-4bec-a324-ba83eb80e0be None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:16.895541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457ea9d-8566-4bec-a324-ba83eb80e0be None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:16.895810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457ea9d-8566-4bec-a324-ba83eb80e0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:16.906148 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:16.927165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2457ea9d-8566-4bec-a324-ba83eb80e0be None None] MFA Rules not processed for user `70046c1f26334c8a960acd8f301549c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:16.932893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2457ea9d-8566-4bec-a324-ba83eb80e0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:17.012737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 995/2023] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:17.017703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:17.080663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3416a91-7376-4f9e-9e03-e3080c4693bf admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:17.085424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:17.086066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.086402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.086672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.087298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:17.089173 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:17.095781 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:17.221816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1029/2024] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:17 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:17.225757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457ea9d-8566-4bec-a324-ba83eb80e0be None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:17.237200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457ea9d-8566-4bec-a324-ba83eb80e0be None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:17.291277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:17.291930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.292209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.292480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.293099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:17.295025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:17.316893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:17.374122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 996/2025] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:17 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:17.378112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:17.385880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:17.389723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98bf1c-9c76-4810-a745-61e5c8d7c1c1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:17.394290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:17.394885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.395166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.395439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.396108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:17.397732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:17.408987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1030/2026] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:17.413055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:17.423765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8ecc7-0862-48ec-b59d-dc454736945d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:17.428192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:17.428832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.429109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.429377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] PATH_INFO: `/v3/projects/8279e17c4a0f41ec90a845ae8bc6e7ad/users/d16809e569064e7880bee24fe58d82f3/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.444392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8279e17c4a0f41ec90a845ae8bc6e7ad, user_id=d16809e569064e7880bee24fe58d82f3, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:17.447225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:17.558203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 997/2027] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:17 2022] PUT /identity/v3/projects/8279e17c4a0f41ec90a845ae8bc6e7ad/users/d16809e569064e7880bee24fe58d82f3/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:17.563629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:17.573278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547fbd2-8f55-4665-9023-53af19d07ca7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:17.612333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:17.629957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1643021-900d-4d45-8114-b11b49eeb560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:17.630616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1643021-900d-4d45-8114-b11b49eeb560 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.630909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1643021-900d-4d45-8114-b11b49eeb560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.631180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1643021-900d-4d45-8114-b11b49eeb560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.631825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1643021-900d-4d45-8114-b11b49eeb560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:17.632543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:17.633944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1643021-900d-4d45-8114-b11b49eeb560 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:17.643543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1031/2028] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:17.647631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:17.658502 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a50153-fcd7-45e0-a3bd-82a6d998eb4c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:17.663286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:17.663942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.665439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.665439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] PATH_INFO: `/v3/projects/8279e17c4a0f41ec90a845ae8bc6e7ad/users/d16809e569064e7880bee24fe58d82f3/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.666640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8279e17c4a0f41ec90a845ae8bc6e7ad, user_id=d16809e569064e7880bee24fe58d82f3, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:17.669268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:17.679720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 998/2029] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:17 2022] PUT /identity/v3/projects/8279e17c4a0f41ec90a845ae8bc6e7ad/users/d16809e569064e7880bee24fe58d82f3/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:17.690608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aff6a-af4d-4ba5-a1ee-29d0bbdb1df6 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.690895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aff6a-af4d-4ba5-a1ee-29d0bbdb1df6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.691173 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aff6a-af4d-4ba5-a1ee-29d0bbdb1df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.718943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-0b9aff6a-af4d-4ba5-a1ee-29d0bbdb1df6 None None] MFA Rules not processed for user `d16809e569064e7880bee24fe58d82f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:17.720150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0b9aff6a-af4d-4ba5-a1ee-29d0bbdb1df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:17.788245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1032/2030] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:17.798460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:17.800122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:17.856212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab528def-3c7b-4a5b-82fa-e5e78c53cd7a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:17.860080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c80d8-73c6-480d-9316-9039c54e1331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:17.860685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c80d8-73c6-480d-9316-9039c54e1331 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:17.860944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c80d8-73c6-480d-9316-9039c54e1331 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:17.861214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c80d8-73c6-480d-9316-9039c54e1331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:17.871583 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c80d8-73c6-480d-9316-9039c54e1331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:17.873037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c80d8-73c6-480d-9316-9039c54e1331 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:17.882519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 999/2031] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:18.474443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9aff6a-af4d-4ba5-a1ee-29d0bbdb1df6 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:18.479226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:18.536318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9aff6a-af4d-4ba5-a1ee-29d0bbdb1df6 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:18.541378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:18.541992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:18.542289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:18.542561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] PATH_INFO: `/v3/users/70046c1f26334c8a960acd8f301549c5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:18.543715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70046c1f26334c8a960acd8f301549c5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:18.545615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:18.690018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70046c1f26334c8a960acd8f301549c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:18.690693 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70046c1f26334c8a960acd8f301549c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:18.691290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70046c1f26334c8a960acd8f301549c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:18.698313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70046c1f26334c8a960acd8f301549c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:18.768366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70046c1f26334c8a960acd8f301549c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:18.787152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1033/2032] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:18 2022] DELETE /identity/v3/users/70046c1f26334c8a960acd8f301549c5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:18.792433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:18.792878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:18.792973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:18.811400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:18.812553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:18.870535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1000/2033] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:18.880261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:18.881702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:18.938186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdf70c-1210-4853-9001-8a7b3180cb9c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:18.942030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:18.942671 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:18.942953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:18.943225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:18.952581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:18.954004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:18.963957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1034/2034] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:19.393030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:19.394617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:19.406760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec395885-6bf1-4276-9c1f-711659cfd563 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:19.412784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:19.413464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:19.413745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:19.414021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] PATH_INFO: `/v3/projects/47ec6712d1504ee58040e3bf7e904993` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:19.415920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ec6712d1504ee58040e3bf7e904993)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:19.417858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:19.443269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:19.524028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ec6712d1504ee58040e3bf7e904993', 'request_id': 'req-f3ace2d9-d824-4884-83d7-818dc8424b51'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.529464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ec6712d1504ee58040e3bf7e904993', 'request_id': 'req-f3ace2d9-d824-4884-83d7-818dc8424b51'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.535695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ec6712d1504ee58040e3bf7e904993', 'request_id': 'req-f3ace2d9-d824-4884-83d7-818dc8424b51'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.648688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] The token cache is being invalidate because project 47ec6712d1504ee58040e3bf7e904993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:19.649679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.656960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.656960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1001/2035] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:19 2022] DELETE /identity/v3/projects/47ec6712d1504ee58040e3bf7e904993 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:19.659647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:19.669900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c00dcc-b709-4949-a50e-5f6e36cfe661 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:19.689131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:19.726037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:19.726719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:19.727633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:19.727910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] PATH_INFO: `/v3/users/d16809e569064e7880bee24fe58d82f3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:19.729024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d16809e569064e7880bee24fe58d82f3)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:19.730970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:19.966272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd16809e569064e7880bee24fe58d82f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.966918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd16809e569064e7880bee24fe58d82f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.967522 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd16809e569064e7880bee24fe58d82f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:19.975453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd16809e569064e7880bee24fe58d82f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.072454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd16809e569064e7880bee24fe58d82f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.091830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1035/2036] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:19 2022] DELETE /identity/v3/users/d16809e569064e7880bee24fe58d82f3 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:20.559421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:20.562590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:20.577379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ace2d9-d824-4884-83d7-818dc8424b51 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:20.636405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:20.637062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:20.637341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:20.637612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] PATH_INFO: `/v3/projects/8279e17c4a0f41ec90a845ae8bc6e7ad` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:20.638775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8279e17c4a0f41ec90a845ae8bc6e7ad)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:20.640677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:20.684399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:20.685855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:20.707008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16ed2c-59d7-4796-8cd2-a5c26fe960a1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:20.711598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:20.712250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:20.712526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:20.712797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] PATH_INFO: `/v3/users/bf8c785fd79949d0bc49485deab8bdb9` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:20.713917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8c785fd79949d0bc49485deab8bdb9)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:20.715923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:20.747663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8279e17c4a0f41ec90a845ae8bc6e7ad', 'request_id': 'req-402f7cc3-6b6e-4176-b0d0-3b24120bc088'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.753082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8279e17c4a0f41ec90a845ae8bc6e7ad', 'request_id': 'req-402f7cc3-6b6e-4176-b0d0-3b24120bc088'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.759241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8279e17c4a0f41ec90a845ae8bc6e7ad', 'request_id': 'req-402f7cc3-6b6e-4176-b0d0-3b24120bc088'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.813950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] The token cache is being invalidate because project 8279e17c4a0f41ec90a845ae8bc6e7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:20.814260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.814900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-402f7cc3-6b6e-4176-b0d0-3b24120bc088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.816471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1002/2037] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:20 2022] DELETE /identity/v3/projects/8279e17c4a0f41ec90a845ae8bc6e7ad => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:20.821949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f855b-9a52-49cb-a8a9-166fc678b467 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:20.822323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f855b-9a52-49cb-a8a9-166fc678b467 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:20.822574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f855b-9a52-49cb-a8a9-166fc678b467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:20.834801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8c785fd79949d0bc49485deab8bdb9'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.835461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8c785fd79949d0bc49485deab8bdb9'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.836068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8c785fd79949d0bc49485deab8bdb9'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.843475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8c785fd79949d0bc49485deab8bdb9'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.843798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-604f855b-9a52-49cb-a8a9-166fc678b467 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:20.845012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-604f855b-9a52-49cb-a8a9-166fc678b467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:20.883204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:20.911248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1003/2038] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:20.912831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-c809dd3f-40ee-4af8-8156-184a9c0a9f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8c785fd79949d0bc49485deab8bdb9'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:20.920078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c706f-90d7-4051-8e78-d7e3306bff7f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:20.920358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c706f-90d7-4051-8e78-d7e3306bff7f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:20.920624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c706f-90d7-4051-8e78-d7e3306bff7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:20.933084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1036/2039] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:20 2022] DELETE /identity/v3/users/bf8c785fd79949d0bc49485deab8bdb9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:20.937385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:20.937680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:20.937957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:20.940288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-570c706f-90d7-4051-8e78-d7e3306bff7f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:20.942251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-570c706f-90d7-4051-8e78-d7e3306bff7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:20.956858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:20.960197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:20.968562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:21.014260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1004/2040] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:21.029440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c706f-90d7-4051-8e78-d7e3306bff7f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.044171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1037/2041] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:21.045454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c706f-90d7-4051-8e78-d7e3306bff7f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.052386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.053526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.054017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.054529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.056155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:21.056538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.057067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.058619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.058811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:21.075638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1005/2042] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:21.076810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.087074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b6fa4-128f-46d4-89f8-ee77a5891343 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.091816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.092409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.092688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.092957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.093656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.095967 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.123206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716fcbc-2b5d-4e0f-9fca-739321472030 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.131627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.132279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.132561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.132832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.142517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.143884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.154417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1038/2043] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:21.171678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1006/2044] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:21 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:21.175869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.188286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f565c6-5677-45d1-8e01-b7f7ef883043 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.257216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.257890 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.258254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.258530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.259168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.263008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.376708 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1039/2045] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:21 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:21.381162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.392797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0306380-4f4f-4b80-ae9c-df55b2e90a15 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.399109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.399860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.404105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.404105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.404105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.404105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.413765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1007/2046] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:21.429060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.440692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e07e29-1b66-40ba-86af-968dd9451abb admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.445788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.446562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.446908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.447247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.447958 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.449643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.460172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1040/2047] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:21.465671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.475657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7c904-cf6d-495e-a10a-363686d58c0f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.480826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac141-8cd0-42f4-8f11-8dea18dfcd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.481601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac141-8cd0-42f4-8f11-8dea18dfcd85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.481796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac141-8cd0-42f4-8f11-8dea18dfcd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.482073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac141-8cd0-42f4-8f11-8dea18dfcd85 admin admin] PATH_INFO: `/v3/projects/7cd9e68b391940a1bcfe4092b42b9cb5/users/fa598223e6654d288b2e3c2b966e07b7/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.511174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac141-8cd0-42f4-8f11-8dea18dfcd85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cd9e68b391940a1bcfe4092b42b9cb5, user_id=fa598223e6654d288b2e3c2b966e07b7, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.514064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac141-8cd0-42f4-8f11-8dea18dfcd85 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.564938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.566723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:21.587597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca9958-7ae7-45d2-9679-be368bdda097 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.598991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1008/2048] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:21 2022] PUT /identity/v3/projects/7cd9e68b391940a1bcfe4092b42b9cb5/users/fa598223e6654d288b2e3c2b966e07b7/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:21.607436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f2f51-81fb-45f5-be87-a2f8a6ea3f43 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.607809 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f2f51-81fb-45f5-be87-a2f8a6ea3f43 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.608274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f2f51-81fb-45f5-be87-a2f8a6ea3f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.638994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0b9f2f51-81fb-45f5-be87-a2f8a6ea3f43 None None] MFA Rules not processed for user `fa598223e6654d288b2e3c2b966e07b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:21.639600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:21.643086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0b9f2f51-81fb-45f5-be87-a2f8a6ea3f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:21.663808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.665558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.666125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.666759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] PATH_INFO: `/v3/projects/adb6956033134950b8f647a20ef77f56` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.671951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adb6956033134950b8f647a20ef77f56)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.672723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.726277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:21.748264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1009/2049] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:21.749254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f2f51-81fb-45f5-be87-a2f8a6ea3f43 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.767768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f2f51-81fb-45f5-be87-a2f8a6ea3f43 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.788472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adb6956033134950b8f647a20ef77f56', 'request_id': 'req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:21.799309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adb6956033134950b8f647a20ef77f56', 'request_id': 'req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:21.807244 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adb6956033134950b8f647a20ef77f56', 'request_id': 'req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:21.830548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.831242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.831587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.831932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.832685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.834689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.877872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] The token cache is being invalidate because project adb6956033134950b8f647a20ef77f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:21.878343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:21.879589 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:21.880744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1041/2050] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:21 2022] DELETE /identity/v3/projects/adb6956033134950b8f647a20ef77f56 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:21.885110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.886548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:21.904084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c92ce5-4ed7-4a17-a96f-2e923a40ac69 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:21.905286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:21.905910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:21.906215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:21.906513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] PATH_INFO: `/v3/users/894539db640e45aeaa0a0ec9342f64ba` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:21.907643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894539db640e45aeaa0a0ec9342f64ba)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:21.909539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:21.953598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1010/2051] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:21 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:21.957438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:21.959032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:22.028752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8106e04-d466-4b92-ad58-bc1a357f675e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.033821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.034486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.034772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.035051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.035695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.037620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.141336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1011/2052] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:21 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:22.145442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:22.151537 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894539db640e45aeaa0a0ec9342f64ba'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.152288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894539db640e45aeaa0a0ec9342f64ba'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.152928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894539db640e45aeaa0a0ec9342f64ba'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.155733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45560d60-b687-4f69-854c-43bf235802f6 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.160431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.160647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894539db640e45aeaa0a0ec9342f64ba'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.161207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.161468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.161776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.162644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.164719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.175216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1012/2053] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:22.180054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:22.182783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:22.196284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2455e-14fb-4bfc-ad60-9ed337718e90 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.200989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.201657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.201942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.202221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] PATH_INFO: `/v3/projects/031d098201b043e2be61ca31e40116aa/users/f621fb190e2f4a9cb66799e7548d19c5/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.217949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] RBAC: Authorizing `identity:create_grant(project_id=031d098201b043e2be61ca31e40116aa, user_id=f621fb190e2f4a9cb66799e7548d19c5, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.220669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.254236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894539db640e45aeaa0a0ec9342f64ba'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.276169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1042/2054] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:21 2022] DELETE /identity/v3/users/894539db640e45aeaa0a0ec9342f64ba => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:22.325181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1013/2055] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:22 2022] PUT /identity/v3/projects/031d098201b043e2be61ca31e40116aa/users/f621fb190e2f4a9cb66799e7548d19c5/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:22.329906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:22.393520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:22.412685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a10649-f226-4766-bf81-6e0caf935f78 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.478905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f280cff-dc81-4b4a-995e-b307fa07873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.479823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280cff-dc81-4b4a-995e-b307fa07873d admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.479823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280cff-dc81-4b4a-995e-b307fa07873d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.480060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f280cff-dc81-4b4a-995e-b307fa07873d admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.480666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f280cff-dc81-4b4a-995e-b307fa07873d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.482393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f280cff-dc81-4b4a-995e-b307fa07873d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.493417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1043/2056] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:22.499176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:22.517023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9c994-659b-4631-b77a-13ebb7f757db admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.522259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.522936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.523219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.524021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] PATH_INFO: `/v3/projects/031d098201b043e2be61ca31e40116aa/users/f621fb190e2f4a9cb66799e7548d19c5/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.526680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=031d098201b043e2be61ca31e40116aa, user_id=f621fb190e2f4a9cb66799e7548d19c5, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.529437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.543480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1014/2057] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:22 2022] PUT /identity/v3/projects/031d098201b043e2be61ca31e40116aa/users/f621fb190e2f4a9cb66799e7548d19c5/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:22.554737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.555707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.555707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.590748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:22.593098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] MFA Rules not processed for user `f621fb190e2f4a9cb66799e7548d19c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:22.593402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:22.594482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:22.625599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:22.667803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd373a52-b361-416b-a556-73a9f9a8bcfa admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.673282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1044/2058] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:22.673799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.674562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.674847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.675120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] PATH_INFO: `/v3/users/6914b983117c4f0aa3a0365e0798d71e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.676265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6914b983117c4f0aa3a0365e0798d71e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.678493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.686260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:22.687649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:22.745451 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede6532-f647-4361-86ce-a8bf9936c9d2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.749170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.749904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.750111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.750413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.760751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.762361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.772092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1045/2059] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:22.777906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:22.779651 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:22.793406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88203de3-d15b-4859-8c2e-269111ec0c1e service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:22.798213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:22.798865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:22.799141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:22.799411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] PATH_INFO: `/v3/projects/c73d813f16624234af0de2f8e8abc5a0` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:22.800719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c73d813f16624234af0de2f8e8abc5a0)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:22.802697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:22.810370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:22.906135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6914b983117c4f0aa3a0365e0798d71e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.906975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6914b983117c4f0aa3a0365e0798d71e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.907622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6914b983117c4f0aa3a0365e0798d71e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.940952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c73d813f16624234af0de2f8e8abc5a0', 'request_id': 'req-161d26e7-38b8-4c67-b1fa-c476534c3772'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.947328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c73d813f16624234af0de2f8e8abc5a0', 'request_id': 'req-161d26e7-38b8-4c67-b1fa-c476534c3772'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.953581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c73d813f16624234af0de2f8e8abc5a0', 'request_id': 'req-161d26e7-38b8-4c67-b1fa-c476534c3772'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:22.991798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6914b983117c4f0aa3a0365e0798d71e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:23.001951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6914b983117c4f0aa3a0365e0798d71e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:23.025724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1015/2060] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:22 2022] DELETE /identity/v3/users/6914b983117c4f0aa3a0365e0798d71e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:23.026877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.028368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:23.081822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] The token cache is being invalidate because project c73d813f16624234af0de2f8e8abc5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:23.082441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:23.083450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-161d26e7-38b8-4c67-b1fa-c476534c3772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:23.085661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1046/2061] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:22 2022] DELETE /identity/v3/projects/c73d813f16624234af0de2f8e8abc5a0 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:23.087082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680bc84-4be4-44a8-bee5-cf35741ce562 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.087376 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680bc84-4be4-44a8-bee5-cf35741ce562 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.087627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680bc84-4be4-44a8-bee5-cf35741ce562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.092321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852cd86-5ca2-49c5-b748-5b3929698c42 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.096031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.096629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.096905 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.097176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.099200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:23.111690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e680bc84-4be4-44a8-bee5-cf35741ce562 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:23.113171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e680bc84-4be4-44a8-bee5-cf35741ce562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:23.172450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:23.175579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.177380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.185512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1047/2062] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:23.186820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e191bd3-96d1-4b8e-b8a8-ca943171789e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.187105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e191bd3-96d1-4b8e-b8a8-ca943171789e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.187382 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e191bd3-96d1-4b8e-b8a8-ca943171789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.196621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1016/2063] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:23.197969 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.199566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:23.210434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2e191bd3-96d1-4b8e-b8a8-ca943171789e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:23.217979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2e191bd3-96d1-4b8e-b8a8-ca943171789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:23.229048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1048/2064] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:23.232013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13eb03-86a8-4f37-84b0-b5a7cee6aa08 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.232387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13eb03-86a8-4f37-84b0-b5a7cee6aa08 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.232775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13eb03-86a8-4f37-84b0-b5a7cee6aa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.255170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-cc13eb03-86a8-4f37-84b0-b5a7cee6aa08 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:23.256390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cc13eb03-86a8-4f37-84b0-b5a7cee6aa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:23.264861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1049/2065] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:23.268451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9cbd9-61d0-4d7f-ad60-39221663390b service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.268956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13eb03-86a8-4f37-84b0-b5a7cee6aa08 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.275341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.275341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.275341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.275341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.279680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13eb03-86a8-4f37-84b0-b5a7cee6aa08 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.284250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.285649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.285820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.286315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.286604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.286872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.287529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:23.287832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.289540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.298596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1017/2066] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:23.302186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1050/2067] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:23.313484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.325455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf1357-fa90-4628-83df-82b880560af3 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.331351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.331976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.332260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.332589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.333305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.335620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.454462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1018/2068] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:23 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:23.458576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.473257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f050b2-0dbe-4559-b51f-ba0a28a558df admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.562856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.563554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.563859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.564146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.567471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.573714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.639858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:23.648875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1051/2069] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:23 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:23.650518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.664735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3a274-253f-44ac-b71f-067b3521a18e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.680667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.681308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.682039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.682344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.683187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.684968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.737828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1019/2070] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:23.742615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.755946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d02a3-e6b8-4a5a-932a-13f2ca20731d admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.762193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.763727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.764228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.764745 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.765959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.768101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.786458 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1052/2071] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:23.787481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.802158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae65ed-43cb-4b16-ae1b-4e4014a7be0f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.811301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.812044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.813138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.813426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] PATH_INFO: `/v3/projects/512bb1f77f66408884554171bfaa10b0/users/cc16b54e2f52469b8703c9955dfa83fe/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.833731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512bb1f77f66408884554171bfaa10b0, user_id=cc16b54e2f52469b8703c9955dfa83fe, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.836848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.922769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.924224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:23.941165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac08f8-82bd-4d82-8985-f3b162f223e6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:23.946003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:23.946675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:23.946956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:23.947235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] PATH_INFO: `/v3/users/fa598223e6654d288b2e3c2b966e07b7` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:23.948357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa598223e6654d288b2e3c2b966e07b7)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:23.950268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:23.966279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1020/2072] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:23 2022] PUT /identity/v3/projects/512bb1f77f66408884554171bfaa10b0/users/cc16b54e2f52469b8703c9955dfa83fe/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:23.966279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:23.966279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:24.076692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf4369-bc78-4476-a953-f3afb9f2ed19 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.081538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.082267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.082503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.082854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] PATH_INFO: `/v3/projects/d09edb47b6c64c89b148e65de5bfeee2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.083979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09edb47b6c64c89b148e65de5bfeee2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.086619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.189089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa598223e6654d288b2e3c2b966e07b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.189820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa598223e6654d288b2e3c2b966e07b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.190438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa598223e6654d288b2e3c2b966e07b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.200989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa598223e6654d288b2e3c2b966e07b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.213796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:24.217521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09edb47b6c64c89b148e65de5bfeee2', 'request_id': 'req-535ef0fe-457f-45d2-9de3-8b1a070c195d'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.222849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09edb47b6c64c89b148e65de5bfeee2', 'request_id': 'req-535ef0fe-457f-45d2-9de3-8b1a070c195d'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.229064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09edb47b6c64c89b148e65de5bfeee2', 'request_id': 'req-535ef0fe-457f-45d2-9de3-8b1a070c195d'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.311645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0869a983-7271-4c6f-bf93-67f3f1177850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa598223e6654d288b2e3c2b966e07b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.330065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1053/2073] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:23 2022] DELETE /identity/v3/users/fa598223e6654d288b2e3c2b966e07b7 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:24.331957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac61f3-0a41-4953-bf4a-4f6177c5d64e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.332243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac61f3-0a41-4953-bf4a-4f6177c5d64e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.332521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac61f3-0a41-4953-bf4a-4f6177c5d64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.362632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-33ac61f3-0a41-4953-bf4a-4f6177c5d64e None None] MFA Rules not processed for user `cc16b54e2f52469b8703c9955dfa83fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:24.363405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] The token cache is being invalidate because project d09edb47b6c64c89b148e65de5bfeee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:24.363767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.363895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-33ac61f3-0a41-4953-bf4a-4f6177c5d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:24.364629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-535ef0fe-457f-45d2-9de3-8b1a070c195d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:24.366540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1021/2074] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:23 2022] DELETE /identity/v3/projects/d09edb47b6c64c89b148e65de5bfeee2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:24.368156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.368473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.368932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.389271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:24.390957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:24.428387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1054/2075] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:24.429726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.430625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.430904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.449656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:24.450815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:24.451309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1022/2076] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:24.452015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.453273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:24.465606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1055/2077] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:24.465606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.465606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:24.465942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd41a14-e2ca-4948-9f58-c68be9d504d5 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.470194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.470829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.471107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.471381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.472008 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.473943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.521109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a302a3-4a21-474e-b1c3-22c1c0a18009 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.524909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f45e8-aa65-4085-935e-1a5b8422aec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.525536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f45e8-aa65-4085-935e-1a5b8422aec9 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.525851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f45e8-aa65-4085-935e-1a5b8422aec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.526107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f45e8-aa65-4085-935e-1a5b8422aec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.534922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f45e8-aa65-4085-935e-1a5b8422aec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.536332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f45e8-aa65-4085-935e-1a5b8422aec9 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.546250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1056/2078] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:24.549165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976df4-bcba-497a-96a1-cb21fa5e0f82 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.549165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976df4-bcba-497a-96a1-cb21fa5e0f82 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.549165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5976df4-bcba-497a-96a1-cb21fa5e0f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.567856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-a5976df4-bcba-497a-96a1-cb21fa5e0f82 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:24.569145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a5976df4-bcba-497a-96a1-cb21fa5e0f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:24.577517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1057/2079] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:24.581601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5976df4-bcba-497a-96a1-cb21fa5e0f82 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.590769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5976df4-bcba-497a-96a1-cb21fa5e0f82 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.595517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.596573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.596854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.597124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.597769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:24.598086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.599716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.615640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1058/2080] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:24.616666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.627239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b181f-fc0f-4429-bc9d-1d5079c58a6b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.632041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.632667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.632930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.633209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.633613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1023/2081] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/projects => generated 387 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:24.635252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.635883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.637898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.699508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0d86-ad41-4a7c-86ff-652578637ddc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.704297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.704901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.705187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.705462 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.706091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.708052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.742775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1059/2082] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:24.746807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.814746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7ba9c-82ef-42c0-bf4f-c820385792e2 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.819830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.823726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1024/2083] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:24.823872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.823872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.823872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.823872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.824645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.824786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.835013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33b9ce-5e8f-419d-96f4-ade898e447a8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.839840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.840569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.840848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.841119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.841759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.843392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.854343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1025/2084] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:24.866121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.877429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21357-d31d-4264-b332-9e2cb927dc4b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.883015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.883684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.883966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.884238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] PATH_INFO: `/v3/projects/ba7948204fef43abba7906e573710ecf/users/7611669680bc45a09ec55eb35a4b9800/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.901387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7948204fef43abba7906e573710ecf, user_id=7611669680bc45a09ec55eb35a4b9800, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.904102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.917510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1060/2085] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:24 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:24.921507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.931439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839c8d-13ef-4716-b63b-d1b76cc17ffc admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.936162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.936742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.937020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.937295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.938167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.939810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.949854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1061/2086] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:24.953527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.963596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5c56-a1d2-4f26-8fa0-b0a47959d7ae admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:24.968199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:24.968773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:24.969053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:24.969328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:24.969948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:24.971519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:24.982508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1062/2087] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:24.986748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.995839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1026/2088] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:24 2022] PUT /identity/v3/projects/ba7948204fef43abba7906e573710ecf/users/7611669680bc45a09ec55eb35a4b9800/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:24.995839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:24.995839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:25.002623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8239e-30db-43af-9161-88d26542742b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:25.061753 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:25.072563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928869b-f9b2-4694-a037-904efdb5a211 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:25.075933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:25.076703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.076980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.077266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] PATH_INFO: `/v3/projects/e7e01f19882d4b6eb2498d59aa8ad78f/users/ce5e17020ce0436ca26936c73c4d4514/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.077786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:25.078567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.078757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.079069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] PATH_INFO: `/v3/projects/7cd9e68b391940a1bcfe4092b42b9cb5` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.080275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cd9e68b391940a1bcfe4092b42b9cb5)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:25.082447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:25.095084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e01f19882d4b6eb2498d59aa8ad78f, user_id=ce5e17020ce0436ca26936c73c4d4514, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:25.098048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:25.157430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1063/2089] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:24 2022] PUT /identity/v3/projects/e7e01f19882d4b6eb2498d59aa8ad78f/users/ce5e17020ce0436ca26936c73c4d4514/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:25.161568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:25.191701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cd9e68b391940a1bcfe4092b42b9cb5', 'request_id': 'req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.198940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cd9e68b391940a1bcfe4092b42b9cb5', 'request_id': 'req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.208044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cd9e68b391940a1bcfe4092b42b9cb5', 'request_id': 'req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.243045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7ac98-e984-46dc-a604-c3c90bd2e3b3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:25.309865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] The token cache is being invalidate because project 7cd9e68b391940a1bcfe4092b42b9cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:25.310648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.311509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-864c4c9e-14f6-4f6d-a15f-3432e6edc12d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.311876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:25.312584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.312873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.313143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.313749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1027/2090] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:24 2022] DELETE /identity/v3/projects/7cd9e68b391940a1bcfe4092b42b9cb5 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:25.313853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:25.315390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.315552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:25.316373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.316907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.327835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1064/2091] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:25.328943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:25.331157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:25.339974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:25.341330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:25.346482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eee5a-4df0-47fa-a8ea-480a6321783c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:25.357546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1028/2092] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:25.357546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:25.357546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:25.358605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:25.359639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.359989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.360326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] PATH_INFO: `/v3/users/f621fb190e2f4a9cb66799e7548d19c5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.361668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f621fb190e2f4a9cb66799e7548d19c5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:25.363899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:25.371788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003f2c6-6861-477a-aaff-f9c68232cae0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:25.377162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:25.378184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.378740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.378856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] PATH_INFO: `/v3/projects/ba7948204fef43abba7906e573710ecf/users/7611669680bc45a09ec55eb35a4b9800/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.380959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7948204fef43abba7906e573710ecf, user_id=7611669680bc45a09ec55eb35a4b9800, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:25.384031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:25.395394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1029/2093] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:25 2022] PUT /identity/v3/projects/ba7948204fef43abba7906e573710ecf/users/7611669680bc45a09ec55eb35a4b9800/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:25.405073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:25.405073 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:25.456860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a277b-d3a6-48ce-9697-a1aa5bc4c963 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:25.461966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be2240-683c-4437-9739-513b269bc363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:25.462789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2240-683c-4437-9739-513b269bc363 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.463076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2240-683c-4437-9739-513b269bc363 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.463673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be2240-683c-4437-9739-513b269bc363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.524893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2240-683c-4437-9739-513b269bc363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:25.526333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be2240-683c-4437-9739-513b269bc363 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:25.535964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1030/2094] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:25.537248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.537559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.537947 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.566719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] MFA Rules not processed for user `7611669680bc45a09ec55eb35a4b9800`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:25.568081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:25.577175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f621fb190e2f4a9cb66799e7548d19c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.578045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f621fb190e2f4a9cb66799e7548d19c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.578755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f621fb190e2f4a9cb66799e7548d19c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.585616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f621fb190e2f4a9cb66799e7548d19c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.637201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1031/2095] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:25.647744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:25.649619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:25.688754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f621fb190e2f4a9cb66799e7548d19c5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:25.716595 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1065/2096] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:25 2022] DELETE /identity/v3/users/f621fb190e2f4a9cb66799e7548d19c5 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:25.730099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced82143-cbae-4322-8694-d4bd93de99f3 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:25.759075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:25.786342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:25.787052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:25.787283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:25.787558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:25.860021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:25.861647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:25.873290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1032/2097] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:26.026821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:26.028440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:26.125937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92d91c-7af2-4218-9621-fcbf18c29591 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:26.163049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:26.163049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:26.163049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:26.163049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] PATH_INFO: `/v3/users/cc16b54e2f52469b8703c9955dfa83fe` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:26.164551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc16b54e2f52469b8703c9955dfa83fe)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:26.164551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:26.564709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:26.566322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:26.569876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:26.594432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b553f4-714e-448d-ae07-f8138256ed8f service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:26.599680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:26.600337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:26.600619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:26.600896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] PATH_INFO: `/v3/projects/031d098201b043e2be61ca31e40116aa` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:26.603501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=031d098201b043e2be61ca31e40116aa)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:26.605476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:26.666389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc16b54e2f52469b8703c9955dfa83fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.670687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc16b54e2f52469b8703c9955dfa83fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.670687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc16b54e2f52469b8703c9955dfa83fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.678461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc16b54e2f52469b8703c9955dfa83fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.706322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031d098201b043e2be61ca31e40116aa', 'request_id': 'req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.712577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031d098201b043e2be61ca31e40116aa', 'request_id': 'req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.720725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031d098201b043e2be61ca31e40116aa', 'request_id': 'req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.759072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-64eab1fe-e6c1-4747-bf3c-735114a3f572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc16b54e2f52469b8703c9955dfa83fe'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.783110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1066/2098] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:26 2022] DELETE /identity/v3/users/cc16b54e2f52469b8703c9955dfa83fe => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:26.787742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:26.788908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:26.789557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:26.813282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:26.814955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:26.823426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] The token cache is being invalidate because project 031d098201b043e2be61ca31e40116aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:26.825932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.825932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-43afe0b9-6dba-4f31-ab41-6f88ab70be4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:26.826421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1033/2099] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:26 2022] DELETE /identity/v3/projects/031d098201b043e2be61ca31e40116aa => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:26.847071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8f3f-2b23-4a30-a843-02a971a6efc9 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:26.847296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8f3f-2b23-4a30-a843-02a971a6efc9 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:26.847475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8f3f-2b23-4a30-a843-02a971a6efc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:26.870854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-78bd8f3f-2b23-4a30-a843-02a971a6efc9 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:26.872088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-78bd8f3f-2b23-4a30-a843-02a971a6efc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:26.905065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1067/2100] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:26.925083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:26.927668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:26.940520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1034/2101] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:26.948995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6456-f49d-4e1b-8b5e-3b9ce1e309ab None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:26.949817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6456-f49d-4e1b-8b5e-3b9ce1e309ab None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:26.949817 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6456-f49d-4e1b-8b5e-3b9ce1e309ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:26.970057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-147e6456-f49d-4e1b-8b5e-3b9ce1e309ab None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:26.971628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-147e6456-f49d-4e1b-8b5e-3b9ce1e309ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:26.980157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1035/2102] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:26.985664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e6456-f49d-4e1b-8b5e-3b9ce1e309ab None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:26.997327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcd41f-b995-4f83-a27b-19ba751014f3 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.003279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.007311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.007311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.007621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e6456-f49d-4e1b-8b5e-3b9ce1e309ab None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.008312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.018166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.019104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.019463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.019827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.020554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.020683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:27.021049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.022204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.022864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.043929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1068/2103] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:27.045267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.045682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1036/2104] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:27.057970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb570ac-af7c-4d6c-ada1-005d5f177cf8 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.064744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.065065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.065344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.065675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.066738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.070016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.222277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1069/2105] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:27 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:27.225662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.242623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:27.247130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f2bdd-c846-478e-aa9a-d5e6380b7704 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.313849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:27.338223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.338223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.338223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.339293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.339293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.342167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.445657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1037/2106] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:27 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:27.450532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.461948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa355bc8-3a94-4ae1-ab47-a0dfd43975c2 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.467015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.467884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.468396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.468883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.469686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.471532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.487737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1070/2107] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:27.488965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.504303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f774b0b-744d-440f-af43-a0502cb7e022 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.513200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.514098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.514684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.514993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.515638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.517286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.528699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1038/2108] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:27.533125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.544565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df154a-6252-4b6b-b1fd-4c5ee51888bc admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.550306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc1b8b-98e0-4eef-8b38-61d4817d786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.550306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc1b8b-98e0-4eef-8b38-61d4817d786e admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.550306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc1b8b-98e0-4eef-8b38-61d4817d786e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.550723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc1b8b-98e0-4eef-8b38-61d4817d786e admin admin] PATH_INFO: `/v3/projects/ae5e634596cf43648bba0666ec59a578/users/cfcb3b1d38ca4ac4823f801edf2197b1/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.566391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc1b8b-98e0-4eef-8b38-61d4817d786e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5e634596cf43648bba0666ec59a578, user_id=cfcb3b1d38ca4ac4823f801edf2197b1, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.569530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc1b8b-98e0-4eef-8b38-61d4817d786e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.582710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.584901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:27.612487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88091b5-4ad0-4984-a5b2-d935b33162c2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.618886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:27.619637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.619959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.620281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] PATH_INFO: `/v3/projects/512bb1f77f66408884554171bfaa10b0` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.623016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=512bb1f77f66408884554171bfaa10b0)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:27.625531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:27.694459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1071/2109] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:27 2022] PUT /identity/v3/projects/ae5e634596cf43648bba0666ec59a578/users/cfcb3b1d38ca4ac4823f801edf2197b1/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:27.706697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:27.706989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:27.707256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:27.736724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] MFA Rules not processed for user `cfcb3b1d38ca4ac4823f801edf2197b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:27.737971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:27.803795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512bb1f77f66408884554171bfaa10b0', 'request_id': 'req-bb3e2b9c-1725-4f97-b149-b8a179c4f644'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:27.819049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512bb1f77f66408884554171bfaa10b0', 'request_id': 'req-bb3e2b9c-1725-4f97-b149-b8a179c4f644'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:27.825346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1072/2110] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:27.832285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512bb1f77f66408884554171bfaa10b0', 'request_id': 'req-bb3e2b9c-1725-4f97-b149-b8a179c4f644'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:27.851373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.851373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:27.909720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045699d-c39f-4550-89a9-8e0fb8ea984a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:27.934115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] The token cache is being invalidate because project 512bb1f77f66408884554171bfaa10b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:27.934546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:27.937207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:27.938960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1039/2111] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:27 2022] DELETE /identity/v3/projects/512bb1f77f66408884554171bfaa10b0 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:27.942772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:27.945568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:28.006913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:28.008276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:28.008502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:28.011439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:28.013602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:28.030338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2b9c-1725-4f97-b149-b8a179c4f644 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:28.035357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:28.036139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:28.036425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:28.036701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] PATH_INFO: `/v3/users/7611669680bc45a09ec55eb35a4b9800` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:28.037935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7611669680bc45a09ec55eb35a4b9800)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:28.040103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:28.124205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:28.125917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:28.141668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1073/2112] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:28.164650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:28.167235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:28.185644 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa8bdf-4d83-4e14-b2a3-2fb9244b7fda service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:28.192342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:28.193071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:28.193417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:28.193755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] PATH_INFO: `/v3/users/cfcb3b1d38ca4ac4823f801edf2197b1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:28.195587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfcb3b1d38ca4ac4823f801edf2197b1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:28.197665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:28.247637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7611669680bc45a09ec55eb35a4b9800'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.248440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7611669680bc45a09ec55eb35a4b9800'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.250323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7611669680bc45a09ec55eb35a4b9800'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.334115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7611669680bc45a09ec55eb35a4b9800'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.342082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21a26e43-606e-42f5-b5fd-67ea501b70ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7611669680bc45a09ec55eb35a4b9800'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.360411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1040/2113] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:27 2022] DELETE /identity/v3/users/7611669680bc45a09ec55eb35a4b9800 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:28.385996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfcb3b1d38ca4ac4823f801edf2197b1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.386861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfcb3b1d38ca4ac4823f801edf2197b1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.387582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfcb3b1d38ca4ac4823f801edf2197b1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.397892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfcb3b1d38ca4ac4823f801edf2197b1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.535041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfcb3b1d38ca4ac4823f801edf2197b1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:28.556967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1074/2114] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:28 2022] DELETE /identity/v3/users/cfcb3b1d38ca4ac4823f801edf2197b1 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:28.561327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:28.561511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:28.561808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:28.589403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:28.591663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:28.674223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1041/2115] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:28.686418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:28.688140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:28.809967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231dbce-63f8-4930-bb82-7690b5a943c8 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:28.819769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34949bdd-5a6d-4fe5-88e2-59a6ad2d439c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:28.823368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949bdd-5a6d-4fe5-88e2-59a6ad2d439c service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:28.823368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949bdd-5a6d-4fe5-88e2-59a6ad2d439c service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:28.823368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949bdd-5a6d-4fe5-88e2-59a6ad2d439c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:28.842870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34949bdd-5a6d-4fe5-88e2-59a6ad2d439c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:28.846141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34949bdd-5a6d-4fe5-88e2-59a6ad2d439c service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:28.875395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1075/2116] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:29.021427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:29.051622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:29.053357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:29.081963 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc168-8f90-4a10-95ed-232ae3efda6c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:29.096467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:29.097498 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.098574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.098908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] PATH_INFO: `/v3/projects/ba7948204fef43abba7906e573710ecf` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:29.102460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7948204fef43abba7906e573710ecf)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:29.110823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:29.209149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:29.273031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7948204fef43abba7906e573710ecf', 'request_id': 'req-713ab651-9730-4eb2-a2c9-a14150cfbfbe'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.285062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7948204fef43abba7906e573710ecf', 'request_id': 'req-713ab651-9730-4eb2-a2c9-a14150cfbfbe'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.297345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7948204fef43abba7906e573710ecf', 'request_id': 'req-713ab651-9730-4eb2-a2c9-a14150cfbfbe'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.394508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] The token cache is being invalidate because project ba7948204fef43abba7906e573710ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:29.394915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.397061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-713ab651-9730-4eb2-a2c9-a14150cfbfbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.403448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1042/2117] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:29 2022] DELETE /identity/v3/projects/ba7948204fef43abba7906e573710ecf => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:29.408464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2810-1d82-4b7b-83db-5e0b95f81afa None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.408681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2810-1d82-4b7b-83db-5e0b95f81afa None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.409124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd2810-1d82-4b7b-83db-5e0b95f81afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:29.440655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-fafd2810-1d82-4b7b-83db-5e0b95f81afa None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:29.441959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-fafd2810-1d82-4b7b-83db-5e0b95f81afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:29.478470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:29.575296 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1076/2118] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:29.583402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.584070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.584348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:29.606530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:29.607750 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:29.617079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1043/2119] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:29.621634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd2810-1d82-4b7b-83db-5e0b95f81afa None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:29.638601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd2810-1d82-4b7b-83db-5e0b95f81afa None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:29.638601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:29.638601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.638601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.639228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:29.639228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:29.639487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:29.641084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:29.657291 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:29.658766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:29.670360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1077/2120] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:05:29.672592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708fb0a-931a-4034-b1e9-68538e701a22 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:29.675523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:29.682798 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:29.683575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.684022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.684716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] PATH_INFO: `/v3/projects/ae5e634596cf43648bba0666ec59a578` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:29.686475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae5e634596cf43648bba0666ec59a578)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:29.689188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:29.691636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440321-52ce-4c22-9183-3ba60043d5f5 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:29.701949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:29.702614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.702900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.703180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:29.703820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:29.705895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:29.849339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1078/2121] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:29 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:29.850582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:29.874113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae5e634596cf43648bba0666ec59a578', 'request_id': 'req-050d6d72-2150-47de-9864-e577d1022930'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.883674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae5e634596cf43648bba0666ec59a578', 'request_id': 'req-050d6d72-2150-47de-9864-e577d1022930'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.895468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae5e634596cf43648bba0666ec59a578', 'request_id': 'req-050d6d72-2150-47de-9864-e577d1022930'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.950409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60ddec-b03f-40dd-bf45-78c4369ba946 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:29.955809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:29.956452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.956733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.957008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:29.957659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:29.960002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:29.975393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] The token cache is being invalidate because project ae5e634596cf43648bba0666ec59a578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:29.975692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.979696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-050d6d72-2150-47de-9864-e577d1022930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:29.983537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1044/2122] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:29 2022] DELETE /identity/v3/projects/ae5e634596cf43648bba0666ec59a578 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:29.986865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80a660-3a31-4775-9f0d-6cdbc706b88e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:29.987241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80a660-3a31-4775-9f0d-6cdbc706b88e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:29.987605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80a660-3a31-4775-9f0d-6cdbc706b88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.058647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-7a80a660-3a31-4775-9f0d-6cdbc706b88e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:30.067641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7a80a660-3a31-4775-9f0d-6cdbc706b88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:30.093000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1079/2123] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:29 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:30.097114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.100648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:30.110971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1045/2124] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:30.123866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae854fc-37da-483e-87e7-6d0d36580a99 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.127708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae854fc-37da-483e-87e7-6d0d36580a99 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.128535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae854fc-37da-483e-87e7-6d0d36580a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.155806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-cae854fc-37da-483e-87e7-6d0d36580a99 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:30.156032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131457d-278b-4578-8ea6-5a0d664b1fe2 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.157318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cae854fc-37da-483e-87e7-6d0d36580a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:30.172102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1046/2125] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:30.175621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae854fc-37da-483e-87e7-6d0d36580a99 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.178566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.181630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.185576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.195483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.195483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:30.195483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:30.201812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae854fc-37da-483e-87e7-6d0d36580a99 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.206561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.207195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.208280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.208280 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.208960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:30.209334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:30.212792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:30.241483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1047/2126] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:05:30.242858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1080/2127] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:30.243949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.246252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.256352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0ef08-e7b9-4cd4-a2a3-6f562ab57166 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.265080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.268468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.268759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.269042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.271510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:30.286690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:30.286946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1db4a-fd3e-48b1-9927-c5a634ff7854 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.303028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1048/2128] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:30.305658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.306278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.306662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.307190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.307454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.307831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:30.309813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:30.329181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaaaae5-f4b8-4590-a0d3-c953f4633fd4 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.329181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba55b4-ca46-4aee-8535-a68ebd7c9ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.329429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba55b4-ca46-4aee-8535-a68ebd7c9ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.329621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba55b4-ca46-4aee-8535-a68ebd7c9ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.330192 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba55b4-ca46-4aee-8535-a68ebd7c9ea1 admin admin] PATH_INFO: `/v3/projects/292c489810ff42a5a7adbb7e7c251514/users/e8d3e938eea5476991577dab405323ee/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.354350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba55b4-ca46-4aee-8535-a68ebd7c9ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=292c489810ff42a5a7adbb7e7c251514, user_id=e8d3e938eea5476991577dab405323ee, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:30.360849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba55b4-ca46-4aee-8535-a68ebd7c9ea1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:30.479247 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1049/2129] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:30 2022] PUT /identity/v3/projects/292c489810ff42a5a7adbb7e7c251514/users/e8d3e938eea5476991577dab405323ee/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:30.485380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.485721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.486005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.500764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1081/2130] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:30 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:30.512527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.564156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:30.568283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:30.583732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:30.760424 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:30.777093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b5ebb-eb92-4d4f-bc8a-0271d725c1e8 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.795951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.812498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.812498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.812498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.812498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:30.812498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:30.821022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1050/2131] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:30.849217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.851059 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:30.936384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1082/2132] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:30 2022] POST /identity/v3/users => generated 289 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:30.936384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.956384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3476d2-db92-4dd0-91a3-342257b7ec31 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.970808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.970808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.970808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.970808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:30.971153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:30.971153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:30.982897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dc596-fab5-42c2-8ae2-fc5a19d1ef97 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:30.987363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1083/2133] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:30.991053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:30.991323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:30.993205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:30.993635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:30.994050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.004167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5348d-b876-4eaf-bab7-3b34af55dc5a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:31.005143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.006907 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:31.009990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866295c0-6476-4f91-8ae1-a9c4babaace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:31.010825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866295c0-6476-4f91-8ae1-a9c4babaace0 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.011208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866295c0-6476-4f91-8ae1-a9c4babaace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.012096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866295c0-6476-4f91-8ae1-a9c4babaace0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.012862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866295c0-6476-4f91-8ae1-a9c4babaace0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.014713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866295c0-6476-4f91-8ae1-a9c4babaace0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:31.024662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1051/2134] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:31.032845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1084/2135] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:31.037202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:31.049942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc4a33-d91c-4a4e-aae9-84f9a6c3861c service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:31.055928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:31.056603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.056883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.057159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] PATH_INFO: `/v3/projects/e70c0779dd614be2a03ff6109a492b8b/users/118837371e814c1b8fd47837f0022f50/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.073135 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70c0779dd614be2a03ff6109a492b8b, user_id=118837371e814c1b8fd47837f0022f50, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.076043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:31.161837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1052/2136] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:31 2022] PUT /identity/v3/projects/e70c0779dd614be2a03ff6109a492b8b/users/118837371e814c1b8fd47837f0022f50/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:31.173064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6116fa-49e2-4d0c-8e2a-b28725d551b4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.176144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6116fa-49e2-4d0c-8e2a-b28725d551b4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.176144 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6116fa-49e2-4d0c-8e2a-b28725d551b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.217164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7c6116fa-49e2-4d0c-8e2a-b28725d551b4 None None] MFA Rules not processed for user `118837371e814c1b8fd47837f0022f50`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:31.219698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7c6116fa-49e2-4d0c-8e2a-b28725d551b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:31.320239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1085/2137] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:31.324508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:31.385869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:31.402905 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cdf52-90bc-4848-85b2-1e95bc12bd2b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:31.409665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:31.410569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.410569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.410859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.411485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.414672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:31.554637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1053/2138] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:31 2022] POST /identity/v3/projects => generated 377 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:31.556669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6116fa-49e2-4d0c-8e2a-b28725d551b4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:31.571131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6116fa-49e2-4d0c-8e2a-b28725d551b4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:31.648401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:31.649157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.649505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.649850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.650536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.652873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:31.717021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1086/2139] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:31 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:31.724408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:31.737984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004696ec-2830-4bfb-b865-75699ae47a0d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:31.744836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:31.750076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.750076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.750076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.750076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.751024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:31.766040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1054/2140] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:31.770973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:31.791198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338de88-40d8-4a7a-9424-752492dde9c9 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:31.798015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b83d6-f80a-4f04-9eaa-c88778c4a9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:31.798828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b83d6-f80a-4f04-9eaa-c88778c4a9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.799200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b83d6-f80a-4f04-9eaa-c88778c4a9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.799538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b83d6-f80a-4f04-9eaa-c88778c4a9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.800310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b83d6-f80a-4f04-9eaa-c88778c4a9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.802385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b83d6-f80a-4f04-9eaa-c88778c4a9d9 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:31.815291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1087/2141] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:31.819638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:31.849357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bbdb0-2c7f-4ec6-804e-518b9691ff30 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:31.862109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:31.862827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:31.863110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:31.863385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] PATH_INFO: `/v3/projects/ddbbc250333c40e8b5a4c1a4030a400e/users/8bb6d32820cd44f8b48a6c852d09fb61/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:31.911166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbbc250333c40e8b5a4c1a4030a400e, user_id=8bb6d32820cd44f8b48a6c852d09fb61, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:31.916354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:32.000220 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1055/2142] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:31 2022] PUT /identity/v3/projects/ddbbc250333c40e8b5a4c1a4030a400e/users/8bb6d32820cd44f8b48a6c852d09fb61/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:32.074810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18d0da-7ccc-412c-89e7-2eef3815cab2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:32.075214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18d0da-7ccc-412c-89e7-2eef3815cab2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:32.075566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18d0da-7ccc-412c-89e7-2eef3815cab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:32.127790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ef18d0da-7ccc-412c-89e7-2eef3815cab2 None None] MFA Rules not processed for user `8bb6d32820cd44f8b48a6c852d09fb61`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:32.129050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ef18d0da-7ccc-412c-89e7-2eef3815cab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:32.174815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:32.272065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1088/2143] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:32.274741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:32.365476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf72a-6f07-4634-a0e9-0084c9bb9727 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:32.374380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:32.375105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:32.376820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:32.376820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:32.376820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:32.378599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:32.490622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1056/2144] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:32 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:32.490931 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18d0da-7ccc-412c-89e7-2eef3815cab2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:32.502812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18d0da-7ccc-412c-89e7-2eef3815cab2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:32.564687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:32.565562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:32.566022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:32.566480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:32.567265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:32.569199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:32.659745 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1089/2145] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:32 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:32.664043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:32.678153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6089b601-6b3b-4b83-a111-591c5a76252e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:32.684531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:32.685314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:32.685695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:32.686121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:32.687317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:32.688998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:32.701145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1057/2146] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:32.705389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:32.718015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caa474-74ed-4d35-9de8-26739ae48935 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:32.723818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:32.724504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:32.724786 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:32.725062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] PATH_INFO: `/v3/projects/575ec453275140c19b143ceb839ff646/users/16db2df4b99647c28db00917ab594935/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:32.745465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575ec453275140c19b143ceb839ff646, user_id=16db2df4b99647c28db00917ab594935, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:32.748321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:32.809788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:32.947715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:32.951430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1090/2147] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:32 2022] PUT /identity/v3/projects/575ec453275140c19b143ceb839ff646/users/16db2df4b99647c28db00917ab594935/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:32.959643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a885f3-5696-4f54-95e2-f10238febd9a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:33.059217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362f716-7791-4411-9f20-f79892c741e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:33.060003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362f716-7791-4411-9f20-f79892c741e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:33.060383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362f716-7791-4411-9f20-f79892c741e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:33.060789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362f716-7791-4411-9f20-f79892c741e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:33.061680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362f716-7791-4411-9f20-f79892c741e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:33.064177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362f716-7791-4411-9f20-f79892c741e3 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:33.082306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1058/2148] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:33.083603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:33.099068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb524d6-01f2-49cc-be07-5dce88f860c8 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:33.107957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:33.109108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:33.109393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:33.109674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] PATH_INFO: `/v3/projects/575ec453275140c19b143ceb839ff646/users/16db2df4b99647c28db00917ab594935/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:33.111687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575ec453275140c19b143ceb839ff646, user_id=16db2df4b99647c28db00917ab594935, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:33.114534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:33.127964 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1091/2149] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:33 2022] PUT /identity/v3/projects/575ec453275140c19b143ceb839ff646/users/16db2df4b99647c28db00917ab594935/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:33.138326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d1b04-a5c2-47df-b454-876cf32b3599 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:33.138789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d1b04-a5c2-47df-b454-876cf32b3599 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:33.139062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d1b04-a5c2-47df-b454-876cf32b3599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:33.168032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-014d1b04-a5c2-47df-b454-876cf32b3599 None None] MFA Rules not processed for user `16db2df4b99647c28db00917ab594935`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:33.169256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-014d1b04-a5c2-47df-b454-876cf32b3599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:33.290995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1059/2150] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:33.298311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:33.300700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:33.334793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:33.376139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98462623-3681-483a-aae9-b5cd0b0f1629 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:33.380741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d44ea-be11-45d1-9924-a514171e3c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:33.383198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d44ea-be11-45d1-9924-a514171e3c42 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:33.384417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d44ea-be11-45d1-9924-a514171e3c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:33.384417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d44ea-be11-45d1-9924-a514171e3c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:33.414552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:33.452308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d44ea-be11-45d1-9924-a514171e3c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:33.454041 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d44ea-be11-45d1-9924-a514171e3c42 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:33.481043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1092/2151] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:33.780720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38210-07e2-4096-9364-2cedac062bc2 None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:33.781004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38210-07e2-4096-9364-2cedac062bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:33.781276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda38210-07e2-4096-9364-2cedac062bc2 None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:33.782833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1060/2152] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:05:33.792243 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d289d-3a05-4b95-90fe-8df90a40c960 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:33.796489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d289d-3a05-4b95-90fe-8df90a40c960 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:33.796489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d289d-3a05-4b95-90fe-8df90a40c960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:33.840879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ab5d289d-3a05-4b95-90fe-8df90a40c960 None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:33.842637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ab5d289d-3a05-4b95-90fe-8df90a40c960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:33.868442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:33.940510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1093/2153] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:05:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:33.953255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38210-07e2-4096-9364-2cedac062bc2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:33.955231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bda38210-07e2-4096-9364-2cedac062bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:34.001162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda38210-07e2-4096-9364-2cedac062bc2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:34.006788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae3544-71a8-4924-b1c3-2f59807bd41f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:34.007659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae3544-71a8-4924-b1c3-2f59807bd41f service placement] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:34.007938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae3544-71a8-4924-b1c3-2f59807bd41f service placement] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:34.008221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae3544-71a8-4924-b1c3-2f59807bd41f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:34.030277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae3544-71a8-4924-b1c3-2f59807bd41f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:34.031908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae3544-71a8-4924-b1c3-2f59807bd41f service placement] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:34.046304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1061/2154] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:34.048989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:34.049279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:34.049553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:34.095100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] MFA Rules not processed for user `e8d3e938eea5476991577dab405323ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:34.103534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:34.145686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1fc10-a673-4048-b5fe-aee0a305860e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:34.145901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1fc10-a673-4048-b5fe-aee0a305860e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:34.146194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1fc10-a673-4048-b5fe-aee0a305860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:34.198504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c6d1fc10-a673-4048-b5fe-aee0a305860e None None] MFA Rules not processed for user `ce5e17020ce0436ca26936c73c4d4514`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:34.200633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c6d1fc10-a673-4048-b5fe-aee0a305860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:34.224567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1094/2155] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:34.237194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:34.238850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:34.242836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:34.252918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dc06a-a0e2-4813-ab57-c43a60fd16eb None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:34.258167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:34.259063 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:34.259517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:34.259925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:34.274358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:34.276530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:34.299363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1095/2156] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:34.366127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1062/2157] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:34.377073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:34.395587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:34.597599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80ca21-aad1-4df7-b481-a54dfe2286d5 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:34.612029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:34.614014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:34.615399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:34.620371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:34.627495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:34.628357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:34.745341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1096/2158] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:34 2022] POST /identity/v3/projects => generated 371 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:34.750895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1fc10-a673-4048-b5fe-aee0a305860e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:34.768128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1fc10-a673-4048-b5fe-aee0a305860e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:34.826501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:34.881218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:34.882667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:34.884455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:34.884455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:34.884455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:34.892376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:34.977850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1063/2159] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:34 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:34.981883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:34.999504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395cb159-d671-4fcc-a78b-9257fe1e70c6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:35.015950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:35.016660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:35.016953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:35.017226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:35.017981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:35.021917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:35.066789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1097/2160] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:35.068114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:35.090422 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f15fd8-3a5b-4e0f-b3e1-df3f15251e0a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:35.099711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:35.101955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:35.102508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:35.102972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] PATH_INFO: `/v3/projects/8170a3d39ebc4e509d05ee721b3fb8b2/users/02b8d7dc4d084d44b298eb70eabfaee3/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:35.128946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8170a3d39ebc4e509d05ee721b3fb8b2, user_id=02b8d7dc4d084d44b298eb70eabfaee3, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:35.132775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:35.285945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1064/2161] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:35 2022] PUT /identity/v3/projects/8170a3d39ebc4e509d05ee721b3fb8b2/users/02b8d7dc4d084d44b298eb70eabfaee3/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:35.302360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:35.311367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:35.331100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fdc87c-54f4-482e-9437-b8a12284dd2a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:35.448053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:35.448735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:35.449019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:35.449290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:35.449941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:35.453773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:35.490008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1098/2162] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:35.496092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:35.510550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427c0f4-c5f2-429b-a7e0-d16647e26226 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:35.528094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:35.528796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:35.529075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:35.529343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] PATH_INFO: `/v3/projects/8170a3d39ebc4e509d05ee721b3fb8b2/users/02b8d7dc4d084d44b298eb70eabfaee3/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:35.531663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8170a3d39ebc4e509d05ee721b3fb8b2, user_id=02b8d7dc4d084d44b298eb70eabfaee3, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:35.535877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:35.561362 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1065/2163] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:35 2022] PUT /identity/v3/projects/8170a3d39ebc4e509d05ee721b3fb8b2/users/02b8d7dc4d084d44b298eb70eabfaee3/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:35.631729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:35.634702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:35.648721 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21df4f-87a3-4bbf-b861-b70458bd2b6b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:35.715872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:35.716883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:35.717071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:35.717351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] PATH_INFO: `/v3/users/e7fa15fc3ec44a0c85f3c87327565280` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:35.718548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7fa15fc3ec44a0c85f3c87327565280)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:35.720715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:35.729673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:35.852744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:35.956877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7fa15fc3ec44a0c85f3c87327565280'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:35.957976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7fa15fc3ec44a0c85f3c87327565280'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:35.959535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e0dc-7048-47e1-a487-74c7ace5982d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:35.964947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7fa15fc3ec44a0c85f3c87327565280'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:35.969458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ee7b7-17c2-4cb6-8a62-805d97e1bf88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:35.970276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ee7b7-17c2-4cb6-8a62-805d97e1bf88 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:35.970563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ee7b7-17c2-4cb6-8a62-805d97e1bf88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:35.970834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ee7b7-17c2-4cb6-8a62-805d97e1bf88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:35.975547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d65ee7b7-17c2-4cb6-8a62-805d97e1bf88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:35.977399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7fa15fc3ec44a0c85f3c87327565280'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:36.096146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7fa15fc3ec44a0c85f3c87327565280'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:36.128292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ee7b7-17c2-4cb6-8a62-805d97e1bf88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:36.132255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ee7b7-17c2-4cb6-8a62-805d97e1bf88 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:36.142173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1066/2164] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:36.191915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1099/2165] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:35 2022] DELETE /identity/v3/users/e7fa15fc3ec44a0c85f3c87327565280 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:36.243884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:36.508346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79278c80-76fb-4bfc-8cfc-1eb03c90f42a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:36.509321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79278c80-76fb-4bfc-8cfc-1eb03c90f42a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:36.509741 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79278c80-76fb-4bfc-8cfc-1eb03c90f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:36.545838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-79278c80-76fb-4bfc-8cfc-1eb03c90f42a None None] MFA Rules not processed for user `e8d3e938eea5476991577dab405323ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:36.550511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-79278c80-76fb-4bfc-8cfc-1eb03c90f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:36.628504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:36.630147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:36.670822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1067/2166] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:36.781616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5c685-ec05-4c02-8d7e-e710add69557 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:36.790103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:36.791748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:36.792037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:36.792314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] PATH_INFO: `/v3/projects/832d2e1e764c403fbefd075a4a96d5b6` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:36.796140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] RBAC: Authorizing `identity:delete_project(project_id=832d2e1e764c403fbefd075a4a96d5b6)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:36.800885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:36.921854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832d2e1e764c403fbefd075a4a96d5b6', 'request_id': 'req-2e606502-1266-4efc-873b-0635ecae897c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:36.936853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832d2e1e764c403fbefd075a4a96d5b6', 'request_id': 'req-2e606502-1266-4efc-873b-0635ecae897c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:36.944683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832d2e1e764c403fbefd075a4a96d5b6', 'request_id': 'req-2e606502-1266-4efc-873b-0635ecae897c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.038873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] The token cache is being invalidate because project 832d2e1e764c403fbefd075a4a96d5b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:37.039550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.048564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.052959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1100/2167] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:36 2022] DELETE /identity/v3/projects/832d2e1e764c403fbefd075a4a96d5b6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:37.058455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79278c80-76fb-4bfc-8cfc-1eb03c90f42a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:37.083516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79278c80-76fb-4bfc-8cfc-1eb03c90f42a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:37.202738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:37.203856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:37.205392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:37.205704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] PATH_INFO: `/v3/users/79a3479d79d64f349ed7356fb16f3193` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:37.211516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a3479d79d64f349ed7356fb16f3193)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:37.215467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:37.409597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a3479d79d64f349ed7356fb16f3193'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.410474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a3479d79d64f349ed7356fb16f3193'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.411796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a3479d79d64f349ed7356fb16f3193'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.489552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a3479d79d64f349ed7356fb16f3193'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.508828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a3479d79d64f349ed7356fb16f3193'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:37.534338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1068/2168] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:37 2022] DELETE /identity/v3/users/79a3479d79d64f349ed7356fb16f3193 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:38.190918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:38.202611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:38.246303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606502-1266-4efc-873b-0635ecae897c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:38.269711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:38.284348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:38.286160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:38.317697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:38.402053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:38.403050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:38.404081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:38.405939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] PATH_INFO: `/v3/projects/1dc2676bce2043c7adab9397677930d5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:38.409059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc2676bce2043c7adab9397677930d5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:38.413546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:38.437531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506c696-00af-4d75-829d-8dd103f7a6f4 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:38.447143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71584a-1658-459c-85ac-ae3f667ed7f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:38.447785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71584a-1658-459c-85ac-ae3f667ed7f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:38.451031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71584a-1658-459c-85ac-ae3f667ed7f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:38.451031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71584a-1658-459c-85ac-ae3f667ed7f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:38.452464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8a71584a-1658-459c-85ac-ae3f667ed7f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:38.585836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71584a-1658-459c-85ac-ae3f667ed7f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:38.587425 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71584a-1658-459c-85ac-ae3f667ed7f3 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:38.606316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc2676bce2043c7adab9397677930d5', 'request_id': 'req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:38.611251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1069/2169] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:38.634844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc2676bce2043c7adab9397677930d5', 'request_id': 'req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:38.646863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc2676bce2043c7adab9397677930d5', 'request_id': 'req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:38.724138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] The token cache is being invalidate because project 1dc2676bce2043c7adab9397677930d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:38.724433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:38.726689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-11bb5cb5-7fd2-493f-b8f3-3b747981bc42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:38.735711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1101/2170] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:38 2022] DELETE /identity/v3/projects/1dc2676bce2043c7adab9397677930d5 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:38.745442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de44442-8c76-4967-a479-01665d21343e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:38.745908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de44442-8c76-4967-a479-01665d21343e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:38.746275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de44442-8c76-4967-a479-01665d21343e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:38.792707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2de44442-8c76-4967-a479-01665d21343e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:38.795820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2de44442-8c76-4967-a479-01665d21343e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:39.029839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1070/2171] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:39.036460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25017-6e5b-4bc4-89bd-23274a248021 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.036955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25017-6e5b-4bc4-89bd-23274a248021 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.037406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed25017-6e5b-4bc4-89bd-23274a248021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.067568 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-aed25017-6e5b-4bc4-89bd-23274a248021 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:39.071935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-aed25017-6e5b-4bc4-89bd-23274a248021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:39.089339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1102/2172] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:39.093530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de44442-8c76-4967-a479-01665d21343e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.110156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de44442-8c76-4967-a479-01665d21343e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.125795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.127097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.127546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.127973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.128799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:39.129263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.131348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.168622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1071/2173] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:05:39.172954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25017-6e5b-4bc4-89bd-23274a248021 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.188357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed25017-6e5b-4bc4-89bd-23274a248021 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.198734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379c388-3b36-4dd8-bb00-9c86ae0d59e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.199617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379c388-3b36-4dd8-bb00-9c86ae0d59e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.205643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379c388-3b36-4dd8-bb00-9c86ae0d59e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.206114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379c388-3b36-4dd8-bb00-9c86ae0d59e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.207522 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379c388-3b36-4dd8-bb00-9c86ae0d59e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.210725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379c388-3b36-4dd8-bb00-9c86ae0d59e4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.325379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1103/2174] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:39 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:39.325856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.343439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a943020-571d-4fa3-828b-8c674c83b072 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.475839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.475839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.475839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.475839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.476207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.478075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.478211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68d54e-19be-4120-b8a7-fcaba9556012 None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.478509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68d54e-19be-4120-b8a7-fcaba9556012 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.478785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68d54e-19be-4120-b8a7-fcaba9556012 None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.480134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1104/2175] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 22 11:05:39.497514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.497799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.498271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.519166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] MFA Rules not processed for user `f3d2213b387f4938ae20eeb16c9352de`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:39.520795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:39.607433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1105/2176] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:05:39.620357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.622002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:39.637275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824f8f6-e770-42be-85c9-3decc8499d3c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.642561 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.643481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.643761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.648087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.657104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1072/2177] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:39 2022] POST /identity/v3/users => generated 293 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:39.658417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.671385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.673043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.677500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a96e6a-e20b-446e-a1e5-baf4b699e9c8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.686775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1106/2178] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:39.687463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.687648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.688975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.689484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.689653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:39.691145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.693730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.696211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.710550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a78c4-773b-4878-998c-b7aa250be390 service placement] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.722995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:39.726629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.727712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.728586 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.729014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] PATH_INFO: `/v3/users/7acea30e695341c8a1d83f386629a99b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.737412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7acea30e695341c8a1d83f386629a99b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.737412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.776269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1073/2179] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:39.778091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.801155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbff04-fe16-4177-8a23-47010b8082c6 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.807898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.808828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.809264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.809677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.810493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.812348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.842569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1074/2180] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:05:39.845563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:39.870140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a920101-8372-4266-84cb-b4f368ddde9f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:39.884704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06e1dc-ea2c-4b36-86a9-6b0ef5c379aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:39.885695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06e1dc-ea2c-4b36-86a9-6b0ef5c379aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:39.886155 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06e1dc-ea2c-4b36-86a9-6b0ef5c379aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:39.886622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06e1dc-ea2c-4b36-86a9-6b0ef5c379aa admin admin] PATH_INFO: `/v3/projects/58fab2cf3d934a758693ecb6815d6701/users/a1071dd23997452cbf11d462cde3baa6/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:39.911928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06e1dc-ea2c-4b36-86a9-6b0ef5c379aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fab2cf3d934a758693ecb6815d6701, user_id=a1071dd23997452cbf11d462cde3baa6, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:39.915204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06e1dc-ea2c-4b36-86a9-6b0ef5c379aa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:39.988078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7acea30e695341c8a1d83f386629a99b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:39.989242 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7acea30e695341c8a1d83f386629a99b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:39.990077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7acea30e695341c8a1d83f386629a99b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:40.000298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7acea30e695341c8a1d83f386629a99b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:40.030423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1075/2181] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:39 2022] PUT /identity/v3/projects/58fab2cf3d934a758693ecb6815d6701/users/a1071dd23997452cbf11d462cde3baa6/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:40.038735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f32c7-ca57-41df-946e-eee860fa1598 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:40.039310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f32c7-ca57-41df-946e-eee860fa1598 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:40.039932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f32c7-ca57-41df-946e-eee860fa1598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:40.080698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-375f32c7-ca57-41df-946e-eee860fa1598 None None] MFA Rules not processed for user `a1071dd23997452cbf11d462cde3baa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:40.083202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:40.084094 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-375f32c7-ca57-41df-946e-eee860fa1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:40.117221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7acea30e695341c8a1d83f386629a99b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:40.148051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1107/2182] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:39 2022] DELETE /identity/v3/users/7acea30e695341c8a1d83f386629a99b => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:40.207466 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1076/2183] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:40.212712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:40.218975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:40.315488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9968924-284a-4702-8486-0d66d5f04e25 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:40.322672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:40.323722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:40.324183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:40.324634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:40.325510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:40.327873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:40.413654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1108/2184] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:40 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:40.418256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f32c7-ca57-41df-946e-eee860fa1598 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:40.523026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f32c7-ca57-41df-946e-eee860fa1598 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:40.602540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:40.604283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:40.604283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:40.604283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:40.619074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:40.621194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:40.643087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:40.738181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1077/2185] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:40 2022] POST /identity/v3/users => generated 292 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:40.742548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:40.753673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409fff0-bf82-4b1f-8aa4-f97fcd05434a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:40.759949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:40.760738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:40.761232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:40.761603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:40.762380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:40.764402 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:40.793250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:40.802111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1109/2186] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:40.852172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ddf3-37fd-49fa-a462-4e584c3f1530 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:40.860301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc5467-4e7c-46b0-a0e1-dba144daa78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:40.861238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc5467-4e7c-46b0-a0e1-dba144daa78d admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:40.862329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc5467-4e7c-46b0-a0e1-dba144daa78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:40.863479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc5467-4e7c-46b0-a0e1-dba144daa78d admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:40.864307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc5467-4e7c-46b0-a0e1-dba144daa78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:40.869285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc5467-4e7c-46b0-a0e1-dba144daa78d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:40.883579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1078/2187] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:40.888803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:40.901963 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dc1e9-172a-4882-95c4-49a06e438b18 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:40.906719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:40.907395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:40.907676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:40.907958 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] PATH_INFO: `/v3/projects/1e864bad97ea44a78b78819b5ebb19f8/users/b6c4affae4de4bfdbdfb8039d0c88d49/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:40.927780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e864bad97ea44a78b78819b5ebb19f8, user_id=b6c4affae4de4bfdbdfb8039d0c88d49, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:40.931808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.037300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1110/2188] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:40 2022] PUT /identity/v3/projects/1e864bad97ea44a78b78819b5ebb19f8/users/b6c4affae4de4bfdbdfb8039d0c88d49/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:41.048458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d59a6-0b59-4747-b295-ca6102fc1ac7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.048960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d59a6-0b59-4747-b295-ca6102fc1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.049385 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d59a6-0b59-4747-b295-ca6102fc1ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.086301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ac6d59a6-0b59-4747-b295-ca6102fc1ac7 None None] MFA Rules not processed for user `b6c4affae4de4bfdbdfb8039d0c88d49`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:41.088715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ac6d59a6-0b59-4747-b295-ca6102fc1ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:41.215200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1079/2189] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:41.219298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.221101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:41.226866 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d59a6-0b59-4747-b295-ca6102fc1ac7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.332388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf6f9b-cc6a-4711-9caa-9ebd50afff0b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:41.336931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d59a6-0b59-4747-b295-ca6102fc1ac7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:41.340288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:41.341123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.342174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.342621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] PATH_INFO: `/v3/projects/793621196ef245588f21b7982fe651e1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.343468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:41.344184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=793621196ef245588f21b7982fe651e1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:41.344320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.344960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.345336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.346461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:41.346857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.349351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.461794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793621196ef245588f21b7982fe651e1', 'request_id': 'req-a3b4ec5f-d2ac-45e4-810c-995b961963b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:41.468847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793621196ef245588f21b7982fe651e1', 'request_id': 'req-a3b4ec5f-d2ac-45e4-810c-995b961963b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:41.476952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793621196ef245588f21b7982fe651e1', 'request_id': 'req-a3b4ec5f-d2ac-45e4-810c-995b961963b7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:41.556843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] The token cache is being invalidate because project 793621196ef245588f21b7982fe651e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:41.557349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:41.558225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a3b4ec5f-d2ac-45e4-810c-995b961963b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:41.560275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1111/2190] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:41 2022] DELETE /identity/v3/projects/793621196ef245588f21b7982fe651e1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:41.564896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f752b-c70d-433b-98e1-6b256600b870 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.565335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f752b-c70d-433b-98e1-6b256600b870 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.565757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f752b-c70d-433b-98e1-6b256600b870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.579265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1080/2191] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:41 2022] POST /identity/v3/projects => generated 383 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:41.585236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.585236 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:41.592583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-529f752b-c70d-433b-98e1-6b256600b870 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:41.594005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-529f752b-c70d-433b-98e1-6b256600b870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:41.699521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1b1b1-c270-46f9-ba4a-15cdcc1bc8d9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:41.702342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1112/2192] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:41.710524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583592a4-acc3-40e3-9333-07e5e41b5a2d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.710884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583592a4-acc3-40e3-9333-07e5e41b5a2d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.711224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583592a4-acc3-40e3-9333-07e5e41b5a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.711931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:41.715223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.725076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.725076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.725076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:41.725076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.759163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-583592a4-acc3-40e3-9333-07e5e41b5a2d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:41.760499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-583592a4-acc3-40e3-9333-07e5e41b5a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:41.769927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1113/2193] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:41.779727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583592a4-acc3-40e3-9333-07e5e41b5a2d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.792447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583592a4-acc3-40e3-9333-07e5e41b5a2d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:41.799393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:41.800042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.800310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.800604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.802898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:05:41.805329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:41.805329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.812586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1081/2194] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:41 2022] POST /identity/v3/users => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:41.814493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.821839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1114/2195] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:05:41.823779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.836475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43a81c-a83d-464b-9661-f788cfd16553 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:41.839910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca162e13-f12c-4f0d-92cc-21467a308e04 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:41.845959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:41.845959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.845959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.845959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.846587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:41.847114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.847453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.847992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.848208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:41.848751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:41.850705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.852179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.875785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:41.890788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1082/2196] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:41.894684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.907306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e91927-3cae-47da-ab4d-0b585a8320c9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:41.915787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:41.916742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:41.917170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:41.917587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] PATH_INFO: `/v3/projects/1aded36ef0fe47d681673f99f901fec6/users/a76d5e957cbd456c847d10260013f36c/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:41.941432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aded36ef0fe47d681673f99f901fec6, user_id=a76d5e957cbd456c847d10260013f36c, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:41.944805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:41.974247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1115/2197] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:41 2022] POST /identity/v3/projects => generated 355 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:41.977798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:41.986488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:42.026281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1083/2198] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:41 2022] PUT /identity/v3/projects/1aded36ef0fe47d681673f99f901fec6/users/a76d5e957cbd456c847d10260013f36c/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:42.028701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:42.132251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c607e5-59e4-467d-98b7-58eda9dcdd58 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.167212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769eed9-9b3d-473c-aa3e-e2179f24340d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.174745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.175639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.176075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.176536 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.177345 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:42.180108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:42.225810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1084/2199] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:42.230592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:42.247895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.248675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.249013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.249407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.250116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:42.251455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81703edc-ecd6-432b-b5be-ec6e86a6c3b1 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.252603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:42.259338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589baa2b-e517-4522-90ad-28cf3e3021ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.260662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589baa2b-e517-4522-90ad-28cf3e3021ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.260662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589baa2b-e517-4522-90ad-28cf3e3021ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.260989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589baa2b-e517-4522-90ad-28cf3e3021ba admin admin] PATH_INFO: `/v3/projects/1aded36ef0fe47d681673f99f901fec6/users/a76d5e957cbd456c847d10260013f36c/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.264018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589baa2b-e517-4522-90ad-28cf3e3021ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aded36ef0fe47d681673f99f901fec6, user_id=a76d5e957cbd456c847d10260013f36c, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:42.267338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589baa2b-e517-4522-90ad-28cf3e3021ba admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:42.281616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1085/2200] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:42 2022] PUT /identity/v3/projects/1aded36ef0fe47d681673f99f901fec6/users/a76d5e957cbd456c847d10260013f36c/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:42.293213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.293501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.293786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.328296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] MFA Rules not processed for user `a76d5e957cbd456c847d10260013f36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:42.329704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:42.352059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1116/2201] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:41 2022] POST /identity/v3/users => generated 278 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:42.356412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:42.387775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a9a10-b416-48b7-89a1-8359e696de8b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.448791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1086/2202] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:42.456767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:42.461103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:42.499136 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.502298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.502705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.503054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.503793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:42.505532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:42.523769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1117/2203] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:42.526990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:42.542981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9186-6335-4faf-917f-60eb43059de6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.549268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.550098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.550494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.550917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.552117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:42.555550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:42.574415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1118/2204] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:42.577091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01976-f735-4367-9057-fbb23a869e2b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.577588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:42.582868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c8044-afb1-48a1-b2fb-b87006065865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.583541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8044-afb1-48a1-b2fb-b87006065865 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.583818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8044-afb1-48a1-b2fb-b87006065865 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.584091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8044-afb1-48a1-b2fb-b87006065865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.592384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85814c-2f59-40d1-b116-35e4812f8305 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.592740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-432c8044-afb1-48a1-b2fb-b87006065865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:42.599070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.599908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.601188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.601813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] PATH_INFO: `/v3/projects/0b6dc265005e4ccf83fdf6dbe63a27a9/users/b976f498871a4b548933711a02dd2571/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.624517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6dc265005e4ccf83fdf6dbe63a27a9, user_id=b976f498871a4b548933711a02dd2571, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:42.627878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:42.676872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c8044-afb1-48a1-b2fb-b87006065865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:42.679336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c8044-afb1-48a1-b2fb-b87006065865 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:42.707002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1087/2205] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:42.739503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1119/2206] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:42 2022] PUT /identity/v3/projects/0b6dc265005e4ccf83fdf6dbe63a27a9/users/b976f498871a4b548933711a02dd2571/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:42.750336 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c04591-f334-4773-bb75-51a326855036 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.750634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c04591-f334-4773-bb75-51a326855036 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.750910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c04591-f334-4773-bb75-51a326855036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.785285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c6c04591-f334-4773-bb75-51a326855036 None None] MFA Rules not processed for user `b976f498871a4b548933711a02dd2571`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:42.786749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c6c04591-f334-4773-bb75-51a326855036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:42.916848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1088/2207] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:42.918165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:42.992191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527dcaa-e9f1-4292-8b37-e397da55b495 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:42.997826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:42.998514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:42.998826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:42.999054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:42.999777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:43.001892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:43.184137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1120/2208] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:05:42 2022] POST /identity/v3/projects => generated 355 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:05:43.184341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c04591-f334-4773-bb75-51a326855036 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:43.213068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c04591-f334-4773-bb75-51a326855036 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:43.321931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:43.322803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:43.323186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:43.323555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:43.325414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:43.327289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:43.426682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1089/2209] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:05:43 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:05:43.431109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:43.453906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147dbde-52e9-463d-b3af-31b91c9c7541 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:43.460401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:43.461027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:43.461477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:43.462495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:43.463310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:43.465539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:43.488836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1121/2210] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:43.489520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:43.505079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cafef9-278a-4873-be68-fda4176371ba admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:43.516071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:43.517703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:43.518055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:43.519173 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] PATH_INFO: `/v3/projects/9e7121c291844731977d3a349ccc71dd/users/3ea4db8e63f1444fb6a31ea966354680/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:43.581555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7121c291844731977d3a349ccc71dd, user_id=3ea4db8e63f1444fb6a31ea966354680, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:43.585284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:43.718786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1090/2211] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:43 2022] PUT /identity/v3/projects/9e7121c291844731977d3a349ccc71dd/users/3ea4db8e63f1444fb6a31ea966354680/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:43.722879 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:43.743761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069a6e7-d758-47c1-98f0-ad4f23997670 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:43.790621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:43.802255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:43.805655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:43.888951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:43.888951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:43.888951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:43.888951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:43.889892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:43.889892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:43.913570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1122/2212] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:05:43.914295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:43.944780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f992f-7375-48d0-a809-8187c9a3a2a1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:43.961183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b804cd8-4a9b-4699-88d9-3a9e101f33e8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:43.976728 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:43.977389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:43.977670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:43.977954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] PATH_INFO: `/v3/users/118837371e814c1b8fd47837f0022f50` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:43.978966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ceac9-fa56-465d-b5de-72e69ebc4a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:43.979500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=118837371e814c1b8fd47837f0022f50)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:43.980782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ceac9-fa56-465d-b5de-72e69ebc4a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:43.981712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:43.983232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ceac9-fa56-465d-b5de-72e69ebc4a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:43.985432 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ceac9-fa56-465d-b5de-72e69ebc4a9c admin admin] PATH_INFO: `/v3/projects/9e7121c291844731977d3a349ccc71dd/users/3ea4db8e63f1444fb6a31ea966354680/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:43.989484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ceac9-fa56-465d-b5de-72e69ebc4a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7121c291844731977d3a349ccc71dd, user_id=3ea4db8e63f1444fb6a31ea966354680, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:43.995287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ceac9-fa56-465d-b5de-72e69ebc4a9c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:44.048547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1123/2213] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:05:43 2022] PUT /identity/v3/projects/9e7121c291844731977d3a349ccc71dd/users/3ea4db8e63f1444fb6a31ea966354680/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:44.050066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568747-d815-4018-9918-ec011f509f17 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:44.050429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568747-d815-4018-9918-ec011f509f17 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:44.050695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2568747-d815-4018-9918-ec011f509f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:44.093808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e2568747-d815-4018-9918-ec011f509f17 None None] MFA Rules not processed for user `3ea4db8e63f1444fb6a31ea966354680`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:44.095316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e2568747-d815-4018-9918-ec011f509f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:44.187659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1124/2214] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:44.210162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2568747-d815-4018-9918-ec011f509f17 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:44.214577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e2568747-d815-4018-9918-ec011f509f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:44.336334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118837371e814c1b8fd47837f0022f50'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:44.346318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118837371e814c1b8fd47837f0022f50'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:44.347470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118837371e814c1b8fd47837f0022f50'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:44.354329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2568747-d815-4018-9918-ec011f509f17 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:44.360549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079acc41-09a2-45fe-8c8c-4f028208fd3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:44.360549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079acc41-09a2-45fe-8c8c-4f028208fd3d service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:44.360549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079acc41-09a2-45fe-8c8c-4f028208fd3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:44.360549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079acc41-09a2-45fe-8c8c-4f028208fd3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:44.367363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-079acc41-09a2-45fe-8c8c-4f028208fd3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:44.386343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079acc41-09a2-45fe-8c8c-4f028208fd3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:44.388059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079acc41-09a2-45fe-8c8c-4f028208fd3d service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:44.402416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118837371e814c1b8fd47837f0022f50'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:44.403450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1125/2215] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:44.413766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118837371e814c1b8fd47837f0022f50'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:44.427618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:44.442822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1091/2216] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:43 2022] DELETE /identity/v3/users/118837371e814c1b8fd47837f0022f50 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:05:44.447579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:44.448100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:44.448188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:44.470294 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:44.472554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:44.594598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1126/2217] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:44.662945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:44.666743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:44.732394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:44.740821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:44.803391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ea8c0-dc26-4cb3-be5e-36d165127b5d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:44.814074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e2c6a-808d-4aed-87be-291dbb28ae12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:44.814743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e2c6a-808d-4aed-87be-291dbb28ae12 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:44.815022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e2c6a-808d-4aed-87be-291dbb28ae12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:44.815293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e2c6a-808d-4aed-87be-291dbb28ae12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:44.838880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e2c6a-808d-4aed-87be-291dbb28ae12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:44.844454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e2c6a-808d-4aed-87be-291dbb28ae12 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:44.867443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1092/2218] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:44.874868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d277aa7-c48c-4db2-a718-a1d38efc7f9c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:44.879013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:44.879860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:44.880350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:44.880797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:44.883443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:44.967992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:44.969474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:44.979862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1127/2219] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:44.999003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:44.999451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:44.999586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:45.056223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] MFA Rules not processed for user `02b8d7dc4d084d44b298eb70eabfaee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:45.057474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:45.190848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1093/2220] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:45.211780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:45.213317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:45.248487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:45.329012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6785f-d229-4a67-918a-70804b4e4fa9 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:45.336529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:45.337198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:45.337728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:45.338020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:45.340082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:45.530421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:45.532933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:45.552210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1128/2221] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:45.572602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:45.574413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:45.607224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:45.609043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:45.624407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a42b3-0a68-4b8c-a287-77ae926cb6a8 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:45.635496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a74261-50a7-448e-a7d8-b4491c9d7816 service nova] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:45.636457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:45.637121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:45.637542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:45.637671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] PATH_INFO: `/v3/projects/e70c0779dd614be2a03ff6109a492b8b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:45.639069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70c0779dd614be2a03ff6109a492b8b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:45.641329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:45.649005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:45.649856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:45.650220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:45.650603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:45.653019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:45.726246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:45.726246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:45.737608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1129/2222] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:45.816151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70c0779dd614be2a03ff6109a492b8b', 'request_id': 'req-aa881e86-2701-4df7-8779-d12b7ba8d3ca'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:45.822984 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70c0779dd614be2a03ff6109a492b8b', 'request_id': 'req-aa881e86-2701-4df7-8779-d12b7ba8d3ca'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:45.829391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70c0779dd614be2a03ff6109a492b8b', 'request_id': 'req-aa881e86-2701-4df7-8779-d12b7ba8d3ca'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:45.894670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:45.961175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] The token cache is being invalidate because project e70c0779dd614be2a03ff6109a492b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:45.961175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:45.961760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:45.963670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1094/2223] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:45 2022] DELETE /identity/v3/projects/e70c0779dd614be2a03ff6109a492b8b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:45.967415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:45.981886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4983b30-cad1-4368-9593-ae4655a06ed2 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:46.114573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:46.115947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:46.116323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:46.116691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] PATH_INFO: `/v3/users/8bb6d32820cd44f8b48a6c852d09fb61` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:46.118031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb6d32820cd44f8b48a6c852d09fb61)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:46.120681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:46.287870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb6d32820cd44f8b48a6c852d09fb61'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:46.288608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb6d32820cd44f8b48a6c852d09fb61'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:46.289233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb6d32820cd44f8b48a6c852d09fb61'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:46.302539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb6d32820cd44f8b48a6c852d09fb61'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:46.430939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb6d32820cd44f8b48a6c852d09fb61'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:46.442129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:46.475494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1130/2224] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:45 2022] DELETE /identity/v3/users/8bb6d32820cd44f8b48a6c852d09fb61 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:47.048416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:47.052178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:47.080359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881e86-2701-4df7-8779-d12b7ba8d3ca admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:47.171734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:47.172430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:47.173215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:47.173215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] PATH_INFO: `/v3/projects/ddbbc250333c40e8b5a4c1a4030a400e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:47.174857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbbc250333c40e8b5a4c1a4030a400e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:47.176816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:47.279334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbbc250333c40e8b5a4c1a4030a400e', 'request_id': 'req-8e569684-7cae-48f5-8335-dc83b303fac1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.287285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbbc250333c40e8b5a4c1a4030a400e', 'request_id': 'req-8e569684-7cae-48f5-8335-dc83b303fac1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.306413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbbc250333c40e8b5a4c1a4030a400e', 'request_id': 'req-8e569684-7cae-48f5-8335-dc83b303fac1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.387855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] The token cache is being invalidate because project ddbbc250333c40e8b5a4c1a4030a400e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:47.388209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.389051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.390609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1095/2225] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:47 2022] DELETE /identity/v3/projects/ddbbc250333c40e8b5a4c1a4030a400e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:47.394648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:47.396116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:47.410718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd7517-6817-4912-88ae-1f370ebcbeb3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:47.481717 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:47.482403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:47.482691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:47.482969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] PATH_INFO: `/v3/users/16db2df4b99647c28db00917ab594935` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:47.484395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16db2df4b99647c28db00917ab594935)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:47.486503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:47.610768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:47.612614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:47.699081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e569684-7cae-48f5-8335-dc83b303fac1 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:47.704692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:47.705825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:47.706444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:47.706859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:47.710088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:47.756811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16db2df4b99647c28db00917ab594935'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.757474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16db2df4b99647c28db00917ab594935'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.758217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16db2df4b99647c28db00917ab594935'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.767503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16db2df4b99647c28db00917ab594935'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.791126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:47.792611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:47.803207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1096/2226] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:05:47.808191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:47.831161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-537cf708-1c83-413a-9158-935d471db3b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16db2df4b99647c28db00917ab594935'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:47.847620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:47.860584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1131/2227] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:47 2022] DELETE /identity/v3/users/16db2df4b99647c28db00917ab594935 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:48.450663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:48.452181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:48.469012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf08d5f-0f5b-490b-8b98-0d3dff589968 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:48.586214 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:48.587264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:48.587620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:48.587960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] PATH_INFO: `/v3/projects/575ec453275140c19b143ceb839ff646` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:48.589405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575ec453275140c19b143ceb839ff646)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:48.591549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:48.730201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575ec453275140c19b143ceb839ff646', 'request_id': 'req-c9ffec37-fe07-4578-8a45-8917116e0c54'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:48.738628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575ec453275140c19b143ceb839ff646', 'request_id': 'req-c9ffec37-fe07-4578-8a45-8917116e0c54'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:48.747639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575ec453275140c19b143ceb839ff646', 'request_id': 'req-c9ffec37-fe07-4578-8a45-8917116e0c54'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:48.882505 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] The token cache is being invalidate because project 575ec453275140c19b143ceb839ff646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:48.882847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:48.883515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:48.885418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1097/2228] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:48 2022] DELETE /identity/v3/projects/575ec453275140c19b143ceb839ff646 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:48.920864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d7e2c-28ae-47eb-bb0a-302b637e0150 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:48.921412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d7e2c-28ae-47eb-bb0a-302b637e0150 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:48.921412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d7e2c-28ae-47eb-bb0a-302b637e0150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:48.944235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c82d7e2c-28ae-47eb-bb0a-302b637e0150 None None] MFA Rules not processed for user `ce5e17020ce0436ca26936c73c4d4514`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:48.945604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c82d7e2c-28ae-47eb-bb0a-302b637e0150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:49.058401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1132/2229] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:49.521692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:50.225673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:50.270708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:50.847523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:52.619288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:52.620837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:52.697677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ffec37-fe07-4578-8a45-8917116e0c54 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:52.702366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:52.703004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:52.703285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:52.703559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] PATH_INFO: `/v3/users/a1071dd23997452cbf11d462cde3baa6` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:52.704681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1071dd23997452cbf11d462cde3baa6)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:52.706664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:52.881547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:52.891351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1071dd23997452cbf11d462cde3baa6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:52.892010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1071dd23997452cbf11d462cde3baa6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:52.892759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1071dd23997452cbf11d462cde3baa6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:52.949869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1071dd23997452cbf11d462cde3baa6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:52.957587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1071dd23997452cbf11d462cde3baa6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:52.979985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1098/2230] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:52 2022] DELETE /identity/v3/users/a1071dd23997452cbf11d462cde3baa6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:52.986513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:52.986959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:52.987337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:53.008270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:05:53.009715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:53.070467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1133/2231] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:05:53.080344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:53.081773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:53.143539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b6fed-1689-444f-a3e3-dd492ae3e650 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:53.147544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:53.148350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:53.148638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:53.148934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:53.158890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:53.160357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:53.174768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1099/2232] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:05:53.635925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:53.635925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:53.670815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6a853-2684-4a1d-8d7d-a1654d42d1a2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:53.673913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:53.674726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:53.675075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:53.675357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] PATH_INFO: `/v3/projects/58fab2cf3d934a758693ecb6815d6701` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:53.676479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=58fab2cf3d934a758693ecb6815d6701)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:53.678803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:53.798770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58fab2cf3d934a758693ecb6815d6701', 'request_id': 'req-e9cdcec3-7308-4f96-970a-62ccc59591fd'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:53.804270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58fab2cf3d934a758693ecb6815d6701', 'request_id': 'req-e9cdcec3-7308-4f96-970a-62ccc59591fd'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:53.810633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58fab2cf3d934a758693ecb6815d6701', 'request_id': 'req-e9cdcec3-7308-4f96-970a-62ccc59591fd'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:53.813000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:53.892960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] The token cache is being invalidate because project 58fab2cf3d934a758693ecb6815d6701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:53.893160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:53.893775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:53.895860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1134/2233] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:53 2022] DELETE /identity/v3/projects/58fab2cf3d934a758693ecb6815d6701 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:53.901203 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:53.911268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a923-f733-481c-ba5a-e2ed69ba8d04 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:53.971276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:53.975396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:53.975396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:53.975396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] PATH_INFO: `/v3/users/b6c4affae4de4bfdbdfb8039d0c88d49` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:53.975396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c4affae4de4bfdbdfb8039d0c88d49)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:53.975952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:54.268609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c4affae4de4bfdbdfb8039d0c88d49'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:54.269289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c4affae4de4bfdbdfb8039d0c88d49'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:54.269882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c4affae4de4bfdbdfb8039d0c88d49'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:54.349611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c4affae4de4bfdbdfb8039d0c88d49'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:54.356660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:54.359689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c4affae4de4bfdbdfb8039d0c88d49'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:54.378142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1100/2234] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:53 2022] DELETE /identity/v3/users/b6c4affae4de4bfdbdfb8039d0c88d49 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:54.900826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:54.902315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:54.915751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdcec3-7308-4f96-970a-62ccc59591fd admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:54.980203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:54.980910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:54.981199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:54.981477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] PATH_INFO: `/v3/projects/1e864bad97ea44a78b78819b5ebb19f8` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:54.982718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e864bad97ea44a78b78819b5ebb19f8)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:54.984894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:55.091050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e864bad97ea44a78b78819b5ebb19f8', 'request_id': 'req-e114eefd-0c09-4892-a4cc-4ab1c3c81157'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.097228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e864bad97ea44a78b78819b5ebb19f8', 'request_id': 'req-e114eefd-0c09-4892-a4cc-4ab1c3c81157'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.103532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e864bad97ea44a78b78819b5ebb19f8', 'request_id': 'req-e114eefd-0c09-4892-a4cc-4ab1c3c81157'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.189184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] The token cache is being invalidate because project 1e864bad97ea44a78b78819b5ebb19f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:55.189474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.190106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.192073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1135/2235] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:54 2022] DELETE /identity/v3/projects/1e864bad97ea44a78b78819b5ebb19f8 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:55.196413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:55.198322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:55.221617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb8f41-9c64-4dab-8771-f82077907ebc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:55.323806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:55.324471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:55.324752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:55.325028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] PATH_INFO: `/v3/users/a76d5e957cbd456c847d10260013f36c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:55.326480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76d5e957cbd456c847d10260013f36c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:55.330169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:55.541690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76d5e957cbd456c847d10260013f36c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.542428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76d5e957cbd456c847d10260013f36c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.543084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76d5e957cbd456c847d10260013f36c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.630164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76d5e957cbd456c847d10260013f36c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.642666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-22762127-2060-4ac5-bd3e-996034d88ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76d5e957cbd456c847d10260013f36c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:55.675768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1101/2236] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:05:55 2022] DELETE /identity/v3/users/a76d5e957cbd456c847d10260013f36c => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:05:56.294486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:05:56.295986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:05:56.322388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114eefd-0c09-4892-a4cc-4ab1c3c81157 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:05:56.382258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:05:56.382894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:05:56.383174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:05:56.383447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] PATH_INFO: `/v3/projects/1aded36ef0fe47d681673f99f901fec6` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:05:56.384682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aded36ef0fe47d681673f99f901fec6)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:05:56.386621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:05:56.411416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:INSERT=2,keystone:DELETE=9,keystone:SELECT=89 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:05:56.518413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aded36ef0fe47d681673f99f901fec6', 'request_id': 'req-d147d6c7-175f-4c44-aa7b-6fb739d85465'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:56.525574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aded36ef0fe47d681673f99f901fec6', 'request_id': 'req-d147d6c7-175f-4c44-aa7b-6fb739d85465'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:56.532076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aded36ef0fe47d681673f99f901fec6', 'request_id': 'req-d147d6c7-175f-4c44-aa7b-6fb739d85465'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:56.612066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] The token cache is being invalidate because project 1aded36ef0fe47d681673f99f901fec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:05:56.612358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:56.613000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:05:56.614771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1136/2237] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:05:56 2022] DELETE /identity/v3/projects/1aded36ef0fe47d681673f99f901fec6 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:00.590421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4e32f-38dd-4832-bf69-675976daac0a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:00.596562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4e32f-38dd-4832-bf69-675976daac0a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:00.596562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4e32f-38dd-4832-bf69-675976daac0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:00.611849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-7bb4e32f-38dd-4832-bf69-675976daac0a None None] MFA Rules not processed for user `e8d3e938eea5476991577dab405323ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:00.613711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7bb4e32f-38dd-4832-bf69-675976daac0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:00.695753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:00.703133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1102/2238] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:04.674827 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:04.677688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:04.756070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147d6c7-175f-4c44-aa7b-6fb739d85465 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:04.760841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:04.761699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:04.761977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:04.762273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] PATH_INFO: `/v3/users/b976f498871a4b548933711a02dd2571` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:04.763574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b976f498871a4b548933711a02dd2571)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:04.765651 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:04.958389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b976f498871a4b548933711a02dd2571'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:04.959070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b976f498871a4b548933711a02dd2571'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:04.959735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b976f498871a4b548933711a02dd2571'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:04.967771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b976f498871a4b548933711a02dd2571'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:05.103718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b976f498871a4b548933711a02dd2571'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:05.125563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1137/2239] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:06:04 2022] DELETE /identity/v3/users/b976f498871a4b548933711a02dd2571 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:05.129954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:05.130517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:05.130699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:05.156004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:05.157770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:05.224682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1103/2240] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:05.236974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:05.238431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:05.277287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:05.300632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca104500-1a54-4982-80c2-2e0c4b5bb943 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:05.305121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:05.305737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:05.306014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:05.306316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:05.320530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:05.322593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:05.332685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1138/2241] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:05.829666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:05.831124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:05.851225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b03de-b0c3-4766-ba9b-80ab902c5f67 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:05.869891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:05.870832 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:05.872825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:05.872825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] PATH_INFO: `/v3/projects/0b6dc265005e4ccf83fdf6dbe63a27a9` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:05.873731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6dc265005e4ccf83fdf6dbe63a27a9)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:05.874991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:06.016979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6dc265005e4ccf83fdf6dbe63a27a9', 'request_id': 'req-c327f829-fb5d-48f9-87c3-05791d79e9d2'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.024709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6dc265005e4ccf83fdf6dbe63a27a9', 'request_id': 'req-c327f829-fb5d-48f9-87c3-05791d79e9d2'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.031950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6dc265005e4ccf83fdf6dbe63a27a9', 'request_id': 'req-c327f829-fb5d-48f9-87c3-05791d79e9d2'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.131796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] The token cache is being invalidate because project 0b6dc265005e4ccf83fdf6dbe63a27a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:06:06.132406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.132726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.134273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1104/2242] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:06:05 2022] DELETE /identity/v3/projects/0b6dc265005e4ccf83fdf6dbe63a27a9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:06.138206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:06.148839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88500ee4-a4c4-4918-8855-c78d87449871 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:06.202903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:06.203454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:06.203730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:06.204002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] PATH_INFO: `/v3/users/3ea4db8e63f1444fb6a31ea966354680` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:06.205142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea4db8e63f1444fb6a31ea966354680)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:06.207067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:06.387544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea4db8e63f1444fb6a31ea966354680'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.388232 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea4db8e63f1444fb6a31ea966354680'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.388837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea4db8e63f1444fb6a31ea966354680'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.395854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea4db8e63f1444fb6a31ea966354680'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.468413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9582086d-e69f-474d-a0e1-3553e947053f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea4db8e63f1444fb6a31ea966354680'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:06.487553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1139/2243] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:06:06 2022] DELETE /identity/v3/users/3ea4db8e63f1444fb6a31ea966354680 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:06.921672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:06.923151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:06.938678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327f829-fb5d-48f9-87c3-05791d79e9d2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:07.013200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:07.013945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.014315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.014647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] PATH_INFO: `/v3/projects/9e7121c291844731977d3a349ccc71dd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.015913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7121c291844731977d3a349ccc71dd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:07.017953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:07.127697 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:07.149918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7121c291844731977d3a349ccc71dd', 'request_id': 'req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:07.157418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7121c291844731977d3a349ccc71dd', 'request_id': 'req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:07.165679 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7121c291844731977d3a349ccc71dd', 'request_id': 'req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:07.298027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] The token cache is being invalidate because project 9e7121c291844731977d3a349ccc71dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:06:07.298350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:07.298975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5cc71fc6-12f0-4fea-bd73-36d40835d2c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:07.300565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1105/2244] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:06:06 2022] DELETE /identity/v3/projects/9e7121c291844731977d3a349ccc71dd => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:07.305982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ae29-df5c-49bc-b627-091a7a0fda0a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.306520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ae29-df5c-49bc-b627-091a7a0fda0a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.306613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ae29-df5c-49bc-b627-091a7a0fda0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.328487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-2388ae29-df5c-49bc-b627-091a7a0fda0a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:07.330649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2388ae29-df5c-49bc-b627-091a7a0fda0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:07.382970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:07.395392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1140/2245] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:07.403227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c349861-fdcf-4a7a-b669-49f6932370a5 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.403511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c349861-fdcf-4a7a-b669-49f6932370a5 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.403810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c349861-fdcf-4a7a-b669-49f6932370a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.424459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4c349861-fdcf-4a7a-b669-49f6932370a5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:07.425662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4c349861-fdcf-4a7a-b669-49f6932370a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:07.434344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1106/2246] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:07.438746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ae29-df5c-49bc-b627-091a7a0fda0a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:07.448610 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ae29-df5c-49bc-b627-091a7a0fda0a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:07.453146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:07.453764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.454037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.454357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.455009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:06:07.455296 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:07.457389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:07.469605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1141/2247] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:06:07.474268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c349861-fdcf-4a7a-b669-49f6932370a5 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:07.484774 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c349861-fdcf-4a7a-b669-49f6932370a5 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:07.489329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:07.489921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.490199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.490504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.492758 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:07.495859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:07.602942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1107/2248] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:06:07 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:07.607072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:07.618446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25260b-6c51-4982-aa51-d9f9bb63c6bf admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:07.708002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:07.708002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.708002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.708002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.708318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:07.708318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:07.804415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1142/2249] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:06:07 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:06:07.805613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:07.815745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8f430-4526-4538-8073-2bef50162a1f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:07.821267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:07.822051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.822347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.822622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.823296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:07.824911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:07.837383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1108/2250] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:06:07.841488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:07.858150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6249e8-630f-462a-950f-2145df2b3b15 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:07.864730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fd112-4edb-4f38-8b58-d00886af3eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:07.865354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd112-4edb-4f38-8b58-d00886af3eea admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.865769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd112-4edb-4f38-8b58-d00886af3eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.866585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fd112-4edb-4f38-8b58-d00886af3eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.867315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fd112-4edb-4f38-8b58-d00886af3eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:07.868917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200fd112-4edb-4f38-8b58-d00886af3eea admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:07.880501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1143/2251] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:07.884872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:07.902292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b1f32-f34e-4f1b-8b59-4e3bf5a4253e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:07.902292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:07.902292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:07.902292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:07.902712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] PATH_INFO: `/v3/projects/dd53c84623dc420fb5b9c10f1e6b216d/users/d417304a91cd4a99854ba9a6246826f8/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:07.917852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd53c84623dc420fb5b9c10f1e6b216d, user_id=d417304a91cd4a99854ba9a6246826f8, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:07.920643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:08.054368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1109/2252] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:06:07 2022] PUT /identity/v3/projects/dd53c84623dc420fb5b9c10f1e6b216d/users/d417304a91cd4a99854ba9a6246826f8/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:06:08.065758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f1b1d-453e-41c8-8ed4-d20d34774025 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.066039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f1b1d-453e-41c8-8ed4-d20d34774025 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.066386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f1b1d-453e-41c8-8ed4-d20d34774025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.096733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ef3f1b1d-453e-41c8-8ed4-d20d34774025 None None] MFA Rules not processed for user `d417304a91cd4a99854ba9a6246826f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:08.097951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ef3f1b1d-453e-41c8-8ed4-d20d34774025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:08.165321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1144/2253] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:08.169837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:08.246892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e73a6-5627-4b4e-a34a-0a2f54c72787 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:08.251635 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:08.252320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.252606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.252878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.253514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:08.255768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:08.360416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1110/2254] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:06:08 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:08.361865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f1b1d-453e-41c8-8ed4-d20d34774025 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:08.373991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f1b1d-453e-41c8-8ed4-d20d34774025 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:08.427218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:08.441028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:08.441028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.441028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.441028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.441373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:08.442931 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:08.529629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1145/2255] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:06:08 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:08.530794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:08.541544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce6398-2ecf-4523-9098-92cdc444fb2e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:08.546341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:08.546945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.547221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.547490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.548129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:08.549706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:08.563648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1111/2256] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:08.567676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:08.578419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb4d68-6b18-4357-81cc-433d6104c2d6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:08.583329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e942f9-3b2a-4214-997a-bbdd8dd7b2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:08.583917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e942f9-3b2a-4214-997a-bbdd8dd7b2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.584197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e942f9-3b2a-4214-997a-bbdd8dd7b2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.585081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e942f9-3b2a-4214-997a-bbdd8dd7b2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.585863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e942f9-3b2a-4214-997a-bbdd8dd7b2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:08.587491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e942f9-3b2a-4214-997a-bbdd8dd7b2ea admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:08.597255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1146/2257] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:08.601351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:08.611306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70af63-e80a-4973-87f7-1d2557f90386 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:08.615864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:08.616507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.616782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.617051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] PATH_INFO: `/v3/projects/6a28f2fe9948489ab1cf0379314c5348/users/0a63c023e93849acadfb4a5f0ba17e4e/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.630914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a28f2fe9948489ab1cf0379314c5348, user_id=0a63c023e93849acadfb4a5f0ba17e4e, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:08.634030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:08.727233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:08.732049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1112/2258] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:06:08 2022] PUT /identity/v3/projects/6a28f2fe9948489ab1cf0379314c5348/users/0a63c023e93849acadfb4a5f0ba17e4e/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:08.748213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13c0a3-e562-48f2-8546-f21a121ad3be None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.748514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13c0a3-e562-48f2-8546-f21a121ad3be None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.748796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13c0a3-e562-48f2-8546-f21a121ad3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.777546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7b13c0a3-e562-48f2-8546-f21a121ad3be None None] MFA Rules not processed for user `0a63c023e93849acadfb4a5f0ba17e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:08.778787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7b13c0a3-e562-48f2-8546-f21a121ad3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:08.848043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1147/2259] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:08.853200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:08.917956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cabfc-7dd3-4d3a-a6d0-476ff2113e03 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:08.922802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:08.923418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:08.923683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:08.923958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:08.925519 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:08.927517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:09.034892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1113/2260] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:06:08 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:09.036019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13c0a3-e562-48f2-8546-f21a121ad3be None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:09.051134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13c0a3-e562-48f2-8546-f21a121ad3be None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:09.122182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:09.122814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.123097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.123374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:09.124059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:09.126156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:09.231508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1148/2261] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:06:09 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:09.234633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:09.250757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c73d0d-cd56-4a6f-8e35-f6f9b0db3c64 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:09.258021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:09.258812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.259195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.259690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:09.260638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:09.263923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:09.275064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1114/2262] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:09.279810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:09.291084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a79348-4fa5-4114-bd79-955ee85006cc admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:09.295990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:09.296674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.296956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.297233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] PATH_INFO: `/v3/projects/21c01ac200614deca586ffd87eed833a/users/f10f5ea5be3643f0846c39e9348f045e/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:09.317246 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c01ac200614deca586ffd87eed833a, user_id=f10f5ea5be3643f0846c39e9348f045e, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:09.320093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:09.395919 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1149/2263] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:06:09 2022] PUT /identity/v3/projects/21c01ac200614deca586ffd87eed833a/users/f10f5ea5be3643f0846c39e9348f045e/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:09.400413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:09.414115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354ebca-d82f-4abf-9f05-bcd8006efabb admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:09.485791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe732bcd-22e2-455e-ab2f-ec9626c8b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:09.486577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe732bcd-22e2-455e-ab2f-ec9626c8b9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.486864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe732bcd-22e2-455e-ab2f-ec9626c8b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.487142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe732bcd-22e2-455e-ab2f-ec9626c8b9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:09.487862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe732bcd-22e2-455e-ab2f-ec9626c8b9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:09.489521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe732bcd-22e2-455e-ab2f-ec9626c8b9a4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:09.500977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1115/2264] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:06:09.505083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:09.517515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229c170-7e4c-4b6d-b066-44c1ac804ff7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:09.524091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:09.525227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.525617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.526017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] PATH_INFO: `/v3/projects/21c01ac200614deca586ffd87eed833a/users/f10f5ea5be3643f0846c39e9348f045e/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:09.528137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c01ac200614deca586ffd87eed833a, user_id=f10f5ea5be3643f0846c39e9348f045e, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:09.531770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:09.544151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1150/2265] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:06:09 2022] PUT /identity/v3/projects/21c01ac200614deca586ffd87eed833a/users/f10f5ea5be3643f0846c39e9348f045e/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:06:09.555066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807951-a58b-4f30-8082-8c6349711be0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.555366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807951-a58b-4f30-8082-8c6349711be0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.555727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807951-a58b-4f30-8082-8c6349711be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:09.593623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-74807951-a58b-4f30-8082-8c6349711be0 None None] MFA Rules not processed for user `f10f5ea5be3643f0846c39e9348f045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:09.595089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-74807951-a58b-4f30-8082-8c6349711be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:09.675207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:09.690950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1116/2266] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:09.703164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:09.705339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:09.718790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:09.801181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fd0c4-dd5e-424d-bcf2-06cebc4c189b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:09.807202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a134a75-084f-497e-86a4-7f303e44ed96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:09.808137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134a75-084f-497e-86a4-7f303e44ed96 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.808383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134a75-084f-497e-86a4-7f303e44ed96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.808767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a134a75-084f-497e-86a4-7f303e44ed96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:09.825006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a134a75-084f-497e-86a4-7f303e44ed96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:09.826995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a134a75-084f-497e-86a4-7f303e44ed96 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:09.839728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1151/2267] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:09.972276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807951-a58b-4f30-8082-8c6349711be0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:09.983945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807951-a58b-4f30-8082-8c6349711be0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:09.991893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:09.993012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:09.993012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:09.993449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:10.071943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:10.073522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:10.082416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1117/2268] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:12.758858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e538b10-e58a-431c-a2e6-e399d3f00a06 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:12.758858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e538b10-e58a-431c-a2e6-e399d3f00a06 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:12.758858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e538b10-e58a-431c-a2e6-e399d3f00a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:12.772291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-5e538b10-e58a-431c-a2e6-e399d3f00a06 None None] MFA Rules not processed for user `a1cdc3bfa3e046fa863131d7ff27d4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:12.773509 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5e538b10-e58a-431c-a2e6-e399d3f00a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:12.837469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1152/2269] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:14.452033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:14.516624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153a237-31f7-4c3d-95ee-ac7db6a75305 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:14.521113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:14.522275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:14.522572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:14.522893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] PATH_INFO: `/v3/users/d417304a91cd4a99854ba9a6246826f8` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:14.523976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d417304a91cd4a99854ba9a6246826f8)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:14.525993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:14.751633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd417304a91cd4a99854ba9a6246826f8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:14.754475 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd417304a91cd4a99854ba9a6246826f8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:14.755430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd417304a91cd4a99854ba9a6246826f8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:14.875358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd417304a91cd4a99854ba9a6246826f8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:14.886743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd417304a91cd4a99854ba9a6246826f8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:14.933052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1118/2270] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:06:14 2022] DELETE /identity/v3/users/d417304a91cd4a99854ba9a6246826f8 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:14.935889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:14.936652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:14.937072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:14.966375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:14.968306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:15.087885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1153/2271] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:06:15.094706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:15.096374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:15.113588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:15.172144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd619-bdf8-4f37-a028-8795e47e3bbf admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:15.177069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:15.178526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:15.178942 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:15.179224 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:15.194210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:15.195707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:15.206516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1119/2272] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:15.828712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:15.828712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:15.840316 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d79c3c-308c-4b03-b63b-6ba29d8e3d79 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:15.845630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:15.846182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:15.846490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:15.846780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] PATH_INFO: `/v3/projects/dd53c84623dc420fb5b9c10f1e6b216d` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:15.847877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd53c84623dc420fb5b9c10f1e6b216d)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:15.850797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:15.991815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd53c84623dc420fb5b9c10f1e6b216d', 'request_id': 'req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:15.994844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:16.018558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd53c84623dc420fb5b9c10f1e6b216d', 'request_id': 'req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.029419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd53c84623dc420fb5b9c10f1e6b216d', 'request_id': 'req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.143967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] The token cache is being invalidate because project dd53c84623dc420fb5b9c10f1e6b216d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:06:16.144615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.147394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.149541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1154/2273] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:06:15 2022] DELETE /identity/v3/projects/dd53c84623dc420fb5b9c10f1e6b216d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:16.153360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:16.171170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb006-e785-44d7-9a69-56e7c1f53c04 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:16.276106 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:16.277036 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:16.277606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:16.278057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] PATH_INFO: `/v3/users/0a63c023e93849acadfb4a5f0ba17e4e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:16.279816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a63c023e93849acadfb4a5f0ba17e4e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:16.282493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:16.554630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a63c023e93849acadfb4a5f0ba17e4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.556202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a63c023e93849acadfb4a5f0ba17e4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.556878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a63c023e93849acadfb4a5f0ba17e4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.645610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a63c023e93849acadfb4a5f0ba17e4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.653022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a63c023e93849acadfb4a5f0ba17e4e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:16.669748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1120/2274] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:06:16 2022] DELETE /identity/v3/users/0a63c023e93849acadfb4a5f0ba17e4e => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:17.280167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:17.283552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:17.298590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc15b4c-5d01-4b5c-99e7-016d85532b5e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:17.390137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:17.390944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:17.391323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:17.391695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] PATH_INFO: `/v3/projects/6a28f2fe9948489ab1cf0379314c5348` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:17.393057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a28f2fe9948489ab1cf0379314c5348)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:17.395151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:17.557475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a28f2fe9948489ab1cf0379314c5348', 'request_id': 'req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:17.575187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a28f2fe9948489ab1cf0379314c5348', 'request_id': 'req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:17.593838 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a28f2fe9948489ab1cf0379314c5348', 'request_id': 'req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:17.668457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] The token cache is being invalidate because project 6a28f2fe9948489ab1cf0379314c5348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:06:17.668834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:17.669628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-9b6e3ffd-71d4-4105-b75a-ee937bebd4d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:17.674111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1155/2275] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:06:17 2022] DELETE /identity/v3/projects/6a28f2fe9948489ab1cf0379314c5348 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:17.675607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:17.677667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:17.691060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe8198-5b90-4239-984f-369fa5b2f850 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:17.729037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:17.758547 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:17.759767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:17.760150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:17.760524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] PATH_INFO: `/v3/users/f10f5ea5be3643f0846c39e9348f045e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:17.762332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10f5ea5be3643f0846c39e9348f045e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:17.765137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:18.009198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10f5ea5be3643f0846c39e9348f045e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.009935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10f5ea5be3643f0846c39e9348f045e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.010769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10f5ea5be3643f0846c39e9348f045e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.105728 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10f5ea5be3643f0846c39e9348f045e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.113834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10f5ea5be3643f0846c39e9348f045e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.134822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1121/2276] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:06:17 2022] DELETE /identity/v3/users/f10f5ea5be3643f0846c39e9348f045e => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:18.476801 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5de5-aa9c-41a9-9e5e-d64ff9847c38 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:18.477312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5de5-aa9c-41a9-9e5e-d64ff9847c38 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:18.477630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa5de5-aa9c-41a9-9e5e-d64ff9847c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:18.500025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-83fa5de5-aa9c-41a9-9e5e-d64ff9847c38 None None] MFA Rules not processed for user `7c6fee0b956e41a7b2d9b914043633ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:18.501326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-83fa5de5-aa9c-41a9-9e5e-d64ff9847c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:18.559104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1156/2277] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:18.652945 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:18.654527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:18.729991 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b35f3-18db-4abf-9c23-5533d5dc6f8a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:18.735139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:18.735842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:18.736125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:18.736395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] PATH_INFO: `/v3/projects/21c01ac200614deca586ffd87eed833a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:18.737511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c01ac200614deca586ffd87eed833a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:18.740120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:18.869699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c01ac200614deca586ffd87eed833a', 'request_id': 'req-2b5181b5-0997-4571-b2e0-055b8694eed7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.875324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c01ac200614deca586ffd87eed833a', 'request_id': 'req-2b5181b5-0997-4571-b2e0-055b8694eed7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.883193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c01ac200614deca586ffd87eed833a', 'request_id': 'req-2b5181b5-0997-4571-b2e0-055b8694eed7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.909038 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:18.971808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] The token cache is being invalidate because project 21c01ac200614deca586ffd87eed833a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:06:18.972108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.972765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2b5181b5-0997-4571-b2e0-055b8694eed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:18.981880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1122/2278] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:06:18 2022] DELETE /identity/v3/projects/21c01ac200614deca586ffd87eed833a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:18.983921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88580723-65b9-4bf2-9402-3fda7d1e238f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:18.984859 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88580723-65b9-4bf2-9402-3fda7d1e238f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:18.985143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88580723-65b9-4bf2-9402-3fda7d1e238f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.005652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-88580723-65b9-4bf2-9402-3fda7d1e238f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:19.007093 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-88580723-65b9-4bf2-9402-3fda7d1e238f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:19.034083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:19.074702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1157/2279] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:19.084359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ef669-ff73-4b27-ab18-626d1692ff7b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.085090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ef669-ff73-4b27-ab18-626d1692ff7b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.085333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ef669-ff73-4b27-ab18-626d1692ff7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.113265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-554ef669-ff73-4b27-ab18-626d1692ff7b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:19.120526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-554ef669-ff73-4b27-ab18-626d1692ff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:19.126219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1123/2280] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:19.127531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88580723-65b9-4bf2-9402-3fda7d1e238f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:19.144668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88580723-65b9-4bf2-9402-3fda7d1e238f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:19.149694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:19.150332 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.150619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.150902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.151715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:06:19.152028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:19.154033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:19.172912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1158/2281] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:06:19.174928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ef669-ff73-4b27-ab18-626d1692ff7b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:19.185789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ef669-ff73-4b27-ab18-626d1692ff7b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:19.191601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:19.192289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.192561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.192874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.194108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:19.197006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:19.281506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1124/2282] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:06:19 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:19.286617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:19.302173 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc37e4f-e71c-474d-935b-f174605d1588 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:19.390818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:19.391557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.391936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.392339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.393478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:19.397211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:19.522224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1159/2283] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:06:19 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:19.522941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:19.536508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abb854-319a-4704-a216-16470dc7713d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:19.541383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:19.542004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.542332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.546543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.546543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:19.546543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:19.557027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1125/2284] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:06:19.561100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:19.571743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d10691-544e-4728-b4d7-207657036797 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:19.576913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebef33-2b6f-474b-b86a-d703a7e5c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:19.577517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebef33-2b6f-474b-b86a-d703a7e5c57b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.577802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebef33-2b6f-474b-b86a-d703a7e5c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.578097 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebef33-2b6f-474b-b86a-d703a7e5c57b admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.578793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebef33-2b6f-474b-b86a-d703a7e5c57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:19.580625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebef33-2b6f-474b-b86a-d703a7e5c57b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:19.594272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1160/2285] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:19.598542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:19.608760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791c27-38c0-4d34-9e3c-1161b300e62b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:19.613331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:19.613992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.614302 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.614582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] PATH_INFO: `/v3/projects/336fc8dbf9134f1eae96096df39781b5/users/b40997a030424603bd62c4428eef2d4b/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.629614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=336fc8dbf9134f1eae96096df39781b5, user_id=b40997a030424603bd62c4428eef2d4b, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:19.633310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:19.703613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1126/2286] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:06:19 2022] PUT /identity/v3/projects/336fc8dbf9134f1eae96096df39781b5/users/b40997a030424603bd62c4428eef2d4b/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:06:19.715096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eed00-964f-42dd-89f1-3e19a1097e72 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.715380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eed00-964f-42dd-89f1-3e19a1097e72 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.715652 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eed00-964f-42dd-89f1-3e19a1097e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.744168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f10eed00-964f-42dd-89f1-3e19a1097e72 None None] MFA Rules not processed for user `b40997a030424603bd62c4428eef2d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:19.745333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f10eed00-964f-42dd-89f1-3e19a1097e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:19.814818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1161/2287] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:19.829612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:19.831642 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:19.929867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100717b-e8d8-4105-aec9-bcbea4c8a29a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:19.936672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc959e37-c6ec-4556-8afb-9d6dabf5bac8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:19.937490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc959e37-c6ec-4556-8afb-9d6dabf5bac8 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:19.937776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc959e37-c6ec-4556-8afb-9d6dabf5bac8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:19.938190 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc959e37-c6ec-4556-8afb-9d6dabf5bac8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:19.949503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc959e37-c6ec-4556-8afb-9d6dabf5bac8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:19.951846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc959e37-c6ec-4556-8afb-9d6dabf5bac8 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:19.984240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1127/2288] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:06:19.996258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eed00-964f-42dd-89f1-3e19a1097e72 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:20.126928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eed00-964f-42dd-89f1-3e19a1097e72 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:20.132720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:20.133583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:20.133934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:20.134306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] PATH_INFO: `/v3/users/b40997a030424603bd62c4428eef2d4b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:20.135524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b40997a030424603bd62c4428eef2d4b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:20.137814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:20.188818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:20.305857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b40997a030424603bd62c4428eef2d4b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:20.306616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b40997a030424603bd62c4428eef2d4b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:20.307372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b40997a030424603bd62c4428eef2d4b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:20.319263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b40997a030424603bd62c4428eef2d4b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:20.400602 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b40997a030424603bd62c4428eef2d4b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:20.425047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1162/2289] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:06:19 2022] DELETE /identity/v3/users/b40997a030424603bd62c4428eef2d4b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:20.429662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:20.430010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:20.430397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:20.451604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:20.453499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:20.560292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1128/2290] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:20.571440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:20.576056 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:20.679354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bc7c4-ad26-4636-8b28-f499b0d5877e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:20.685836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35eb19a-e730-4de2-8aa6-fe23ea534c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:20.686558 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35eb19a-e730-4de2-8aa6-fe23ea534c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:20.686836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35eb19a-e730-4de2-8aa6-fe23ea534c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:20.687104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35eb19a-e730-4de2-8aa6-fe23ea534c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:20.699902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35eb19a-e730-4de2-8aa6-fe23ea534c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:20.706429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35eb19a-e730-4de2-8aa6-fe23ea534c38 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:20.712484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1163/2291] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:21.264371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:21.265838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:21.278814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00af21-7bf6-4541-bb34-2f8824a21f5c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:21.284010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:21.284651 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:21.284932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:21.285202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] PATH_INFO: `/v3/projects/336fc8dbf9134f1eae96096df39781b5` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:21.286454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336fc8dbf9134f1eae96096df39781b5)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:21.288890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:21.314507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:21.377663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336fc8dbf9134f1eae96096df39781b5', 'request_id': 'req-5d452760-94e1-456f-a093-fa315b08f1a5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:21.384678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336fc8dbf9134f1eae96096df39781b5', 'request_id': 'req-5d452760-94e1-456f-a093-fa315b08f1a5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:21.391772 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336fc8dbf9134f1eae96096df39781b5', 'request_id': 'req-5d452760-94e1-456f-a093-fa315b08f1a5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:21.478838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] The token cache is being invalidate because project 336fc8dbf9134f1eae96096df39781b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:06:21.479242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:21.479871 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5d452760-94e1-456f-a093-fa315b08f1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:21.481513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1129/2292] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:06:21 2022] DELETE /identity/v3/projects/336fc8dbf9134f1eae96096df39781b5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:21.488783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1f35-02d9-46fc-a112-c4c029723e2e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:21.489075 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1f35-02d9-46fc-a112-c4c029723e2e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:21.489570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1f35-02d9-46fc-a112-c4c029723e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:21.511415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f32f1f35-02d9-46fc-a112-c4c029723e2e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:21.511508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f32f1f35-02d9-46fc-a112-c4c029723e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:21.546215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:21.574961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1164/2293] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:21.583006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049d7e-a842-4e16-966d-deb59b331ad4 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:21.583321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049d7e-a842-4e16-966d-deb59b331ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:21.584226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa049d7e-a842-4e16-966d-deb59b331ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:21.604304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-aa049d7e-a842-4e16-966d-deb59b331ad4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:21.605494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-aa049d7e-a842-4e16-966d-deb59b331ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:21.613835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1130/2294] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:21.618559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f1f35-02d9-46fc-a112-c4c029723e2e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:21.628328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f1f35-02d9-46fc-a112-c4c029723e2e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:21.634763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:21.635371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:21.635654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:21.640333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:21.640333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:06:21.640333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:21.640333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:21.653755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1165/2295] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:06:21.654900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049d7e-a842-4e16-966d-deb59b331ad4 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:21.665301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa049d7e-a842-4e16-966d-deb59b331ad4 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:21.670275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:21.670896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:21.671176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:21.671449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:21.672069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:21.673980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:21.779664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1131/2296] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:06:21 2022] POST /identity/v3/projects => generated 363 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:06:21.787074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:21.797162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84f5cb-c0ca-433b-b283-7ab32e42a143 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:21.859885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:21.860504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:21.860789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:21.861070 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:21.861733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:21.863693 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:21.960783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1166/2297] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:06:21 2022] POST /identity/v3/users => generated 282 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:21.965150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:21.976609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec92be0-ddbd-42dd-9d0d-55af205b8acc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:21.981274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:21.981882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:21.982161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:21.982726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:21.983406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:21.985031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:21.995682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1132/2298] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:22.000079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:22.010641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f73405-d766-464c-bb82-1a4432b64d80 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:22.015714 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da6800-c65b-4d65-a27e-052a3f8195ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:22.016346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da6800-c65b-4d65-a27e-052a3f8195ec admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:22.016765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da6800-c65b-4d65-a27e-052a3f8195ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:22.017051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da6800-c65b-4d65-a27e-052a3f8195ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:22.017777 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da6800-c65b-4d65-a27e-052a3f8195ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:22.019469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da6800-c65b-4d65-a27e-052a3f8195ec admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:22.032333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1167/2299] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:22.037165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:22.047667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a1bc6-3d42-46fd-b38c-74bd2f96e5d1 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:22.052489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:22.053242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:22.053523 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:22.053805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] PATH_INFO: `/v3/projects/0ea4fa264435458cb3d42a5d050bbaa9/users/821cb6d94f7f4b119724c799f8f4a089/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:22.069623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea4fa264435458cb3d42a5d050bbaa9, user_id=821cb6d94f7f4b119724c799f8f4a089, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:22.072194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:22.165293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1133/2300] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:06:22 2022] PUT /identity/v3/projects/0ea4fa264435458cb3d42a5d050bbaa9/users/821cb6d94f7f4b119724c799f8f4a089/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:22.176345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7ecfe-4c98-4721-9d64-b15289cbc502 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:22.176661 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7ecfe-4c98-4721-9d64-b15289cbc502 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:22.176937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7ecfe-4c98-4721-9d64-b15289cbc502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:22.206559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-20e7ecfe-4c98-4721-9d64-b15289cbc502 None None] MFA Rules not processed for user `821cb6d94f7f4b119724c799f8f4a089`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:22.207779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-20e7ecfe-4c98-4721-9d64-b15289cbc502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:22.273391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1168/2301] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:22.285021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:22.287012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:22.346872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828a851-1d8c-4903-8a37-6dec08c8036d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:22.350928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:22.351545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:22.351825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:22.356031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:22.361493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:22.362953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:22.375257 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1134/2302] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:24.262166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2334-e623-40f9-94ed-8ccf2439e091 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:24.263078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2334-e623-40f9-94ed-8ccf2439e091 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:24.263501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2334-e623-40f9-94ed-8ccf2439e091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:24.301698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-77bb2334-e623-40f9-94ed-8ccf2439e091 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:24.303685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-77bb2334-e623-40f9-94ed-8ccf2439e091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:24.355880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:24.461121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1169/2303] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:24.515495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:24.516981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:24.613753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:24.645580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d21b8-2ffa-41fd-ad2c-126cdd4e7fe5 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:24.652654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:24.653977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:24.654299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:24.654583 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:24.667363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:24.670143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:24.681943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1135/2304] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:06:28.972265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:28.972863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:28.972975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:29.001847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:29.003135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:29.015234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1170/2305] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:06:29.031252 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:29.048389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd62d0-6663-426f-a808-d6d749851bb8 service swift] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:29.054323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187439c5-7095-40dd-9499-02f425c6a824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:29.055844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187439c5-7095-40dd-9499-02f425c6a824 service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:29.056327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187439c5-7095-40dd-9499-02f425c6a824 service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:29.056825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187439c5-7095-40dd-9499-02f425c6a824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:29.068623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187439c5-7095-40dd-9499-02f425c6a824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:29.070321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187439c5-7095-40dd-9499-02f425c6a824 service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:29.084785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1136/2306] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:06:35.089394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:35.091424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:35.225877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470d629-84e4-4f86-9ccb-9a6e8f0d0f25 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:35.235932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d694570-41d7-4a94-99f4-dd2ab7c599eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:35.237061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d694570-41d7-4a94-99f4-dd2ab7c599eb service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:35.237552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d694570-41d7-4a94-99f4-dd2ab7c599eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:35.238449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d694570-41d7-4a94-99f4-dd2ab7c599eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:35.241856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9d694570-41d7-4a94-99f4-dd2ab7c599eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:35.370927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d694570-41d7-4a94-99f4-dd2ab7c599eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:35.373797 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d694570-41d7-4a94-99f4-dd2ab7c599eb service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:35.403542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1171/2307] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:39.077204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=17 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:45.389514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=65 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:47.653677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb46d72-efd7-406f-8a3f-920bb6ed070a None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:47.654276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb46d72-efd7-406f-8a3f-920bb6ed070a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:47.654381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb46d72-efd7-406f-8a3f-920bb6ed070a None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:47.655831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1137/2308] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:06:47.668569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d4b6b-1f0b-4c86-a0fb-d7dc7f5d38cf None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:47.669210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d4b6b-1f0b-4c86-a0fb-d7dc7f5d38cf None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:47.669210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d4b6b-1f0b-4c86-a0fb-d7dc7f5d38cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:47.693340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-205d4b6b-1f0b-4c86-a0fb-d7dc7f5d38cf None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:47.694992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-205d4b6b-1f0b-4c86-a0fb-d7dc7f5d38cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:47.783693 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1172/2309] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:53.524627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb46d72-efd7-406f-8a3f-920bb6ed070a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:53.526554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dcb46d72-efd7-406f-8a3f-920bb6ed070a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:53.647669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb46d72-efd7-406f-8a3f-920bb6ed070a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:53.651507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8bd11-bf1f-4cdd-86a9-5c49bfb6e5c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990660ce5a9f4213a5b42b3824d0b567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:53.652230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8bd11-bf1f-4cdd-86a9-5c49bfb6e5c2 service glance] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:53.652539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8bd11-bf1f-4cdd-86a9-5c49bfb6e5c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:53.652909 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8bd11-bf1f-4cdd-86a9-5c49bfb6e5c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:53.669539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8bd11-bf1f-4cdd-86a9-5c49bfb6e5c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:53.671204 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8bd11-bf1f-4cdd-86a9-5c49bfb6e5c2 service glance] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:53.683212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1138/2310] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:06:53.862162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de755f-0c71-4f30-9c04-a9f451acde5e None None] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:53.862685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de755f-0c71-4f30-9c04-a9f451acde5e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:53.862835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de755f-0c71-4f30-9c04-a9f451acde5e None None] PATH_INFO: `/` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:53.864302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1173/2311] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:06:53.871255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb482f5-1fbc-413e-9282-39a47ff29d1c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:53.871550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb482f5-1fbc-413e-9282-39a47ff29d1c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:53.871816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb482f5-1fbc-413e-9282-39a47ff29d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:53.892102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5cb482f5-1fbc-413e-9282-39a47ff29d1c None None] MFA Rules not processed for user `edd00e00b48641c8bcaacaaa7c2dd149`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:53.893295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5cb482f5-1fbc-413e-9282-39a47ff29d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:53.901002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1139/2312] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:53.912730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de755f-0c71-4f30-9c04-a9f451acde5e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:53.914388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-51de755f-0c71-4f30-9c04-a9f451acde5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:53.928491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de755f-0c71-4f30-9c04-a9f451acde5e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:53.932305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4118535-ab0e-4ca1-a130-e07444535e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:53.933053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118535-ab0e-4ca1-a130-e07444535e7f service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:53.933547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118535-ab0e-4ca1-a130-e07444535e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:53.934010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4118535-ab0e-4ca1-a130-e07444535e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:53.943727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4118535-ab0e-4ca1-a130-e07444535e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:53.945377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4118535-ab0e-4ca1-a130-e07444535e7f service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:53.954131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1174/2313] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:54.041328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d0bd-5a91-4ab8-ba6e-6554940e06cc None None] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:54.041617 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d0bd-5a91-4ab8-ba6e-6554940e06cc None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:54.041893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d0bd-5a91-4ab8-ba6e-6554940e06cc None None] PATH_INFO: `/` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:54.043499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1140/2314] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 22 11:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 22 11:06:54.049321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d3778-871c-464b-a4a1-edd44ab7afc6 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:54.049679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d3778-871c-464b-a4a1-edd44ab7afc6 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:54.050028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d3778-871c-464b-a4a1-edd44ab7afc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:54.071341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4b1d3778-871c-464b-a4a1-edd44ab7afc6 None None] MFA Rules not processed for user `7f5fe23bc8294bc3861e0db9dad2045e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:54.072750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4b1d3778-871c-464b-a4a1-edd44ab7afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:54.084574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1175/2315] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 22 11:06:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:06:54.095903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107d0bd-5a91-4ab8-ba6e-6554940e06cc None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:54.107511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107d0bd-5a91-4ab8-ba6e-6554940e06cc None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:54.111052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:54.111584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:54.111861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:54.112128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:54.122920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:54.124368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:54.135778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1141/2316] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:06:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:06:54.693557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24b272-bd8d-4f2e-bf9d-8a234ea5d6cc None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:54.693842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24b272-bd8d-4f2e-bf9d-8a234ea5d6cc None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:54.694118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24b272-bd8d-4f2e-bf9d-8a234ea5d6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:54.719523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-de24b272-bd8d-4f2e-bf9d-8a234ea5d6cc None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:54.725136 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-de24b272-bd8d-4f2e-bf9d-8a234ea5d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:54.728488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1176/2317] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 22 11:06:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:54.742961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:54.755069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c47bf-1a02-41e3-9d17-07e2e3cea596 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:54.760225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:54.760839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:54.765235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:54.765235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:54.770272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:54.772016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:54.781002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1142/2318] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:06:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:06:58.121019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e040a0-80dc-4c70-8824-25064af00887 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:58.121527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e040a0-80dc-4c70-8824-25064af00887 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:58.121728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e040a0-80dc-4c70-8824-25064af00887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:58.142049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e7e040a0-80dc-4c70-8824-25064af00887 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:58.143318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e7e040a0-80dc-4c70-8824-25064af00887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:58.150654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:58.152539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:58.153807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1177/2319] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:58.168565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e040a0-80dc-4c70-8824-25064af00887 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:58.181876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e040a0-80dc-4c70-8824-25064af00887 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:58.187275 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:58.188512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:58.189006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:58.189313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:58.200110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:58.201459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:58.210525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1178/2320] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:06:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 11:06:58.228548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f91c66-b25d-4444-a4c0-cdb7b9d3156c service swift] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:58.233328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:58.233966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:58.234287 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:58.234569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] PATH_INFO: `/v3/users/e8d3e938eea5476991577dab405323ee` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:58.235747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d3e938eea5476991577dab405323ee)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:58.237675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:58.441981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d3e938eea5476991577dab405323ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:58.443253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d3e938eea5476991577dab405323ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:58.446431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d3e938eea5476991577dab405323ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:58.452765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:58.546105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d3e938eea5476991577dab405323ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:58.554709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-5119edbc-3b75-45cb-b72a-408b0bc7d7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d3e938eea5476991577dab405323ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:58.595801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1143/2321] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:06:58 2022] DELETE /identity/v3/users/e8d3e938eea5476991577dab405323ee => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:06:58.915794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:58.935969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff8296-d832-4aed-b556-79a01fd32360 service swift] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:59.046529 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:59.047658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.047934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.048284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] PATH_INFO: `/v3/projects/292c489810ff42a5a7adbb7e7c251514` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.049401 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=292c489810ff42a5a7adbb7e7c251514)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:59.051356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:59.158484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:06:59.164196 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '292c489810ff42a5a7adbb7e7c251514', 'request_id': 'req-566aced7-e4c4-44e6-8bdf-7c846c374eb0'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:59.170369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '292c489810ff42a5a7adbb7e7c251514', 'request_id': 'req-566aced7-e4c4-44e6-8bdf-7c846c374eb0'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:59.178198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '292c489810ff42a5a7adbb7e7c251514', 'request_id': 'req-566aced7-e4c4-44e6-8bdf-7c846c374eb0'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:59.243112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] The token cache is being invalidate because project 292c489810ff42a5a7adbb7e7c251514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:06:59.243399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:59.244359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-566aced7-e4c4-44e6-8bdf-7c846c374eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:06:59.245917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1179/2322] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:06:58 2022] DELETE /identity/v3/projects/292c489810ff42a5a7adbb7e7c251514 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:59.255754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c5d9f-d873-4dbb-b398-8c0e513fea44 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.256341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c5d9f-d873-4dbb-b398-8c0e513fea44 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.256341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c5d9f-d873-4dbb-b398-8c0e513fea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.277571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-688c5d9f-d873-4dbb-b398-8c0e513fea44 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:59.278839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-688c5d9f-d873-4dbb-b398-8c0e513fea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:59.360019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1144/2323] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:59.369446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33d7d7-8dd6-43b9-8f75-ad3391d7cb14 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.369739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33d7d7-8dd6-43b9-8f75-ad3391d7cb14 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.370020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33d7d7-8dd6-43b9-8f75-ad3391d7cb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.392970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9a33d7d7-8dd6-43b9-8f75-ad3391d7cb14 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:59.394209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9a33d7d7-8dd6-43b9-8f75-ad3391d7cb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:59.403676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1180/2324] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:06:59.407742 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c5d9f-d873-4dbb-b398-8c0e513fea44 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:59.418482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c5d9f-d873-4dbb-b398-8c0e513fea44 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:59.423333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:59.423957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.424240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.424515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.425182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:06:59.425489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:59.427966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:59.445790 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1145/2325] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:06:59.446863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33d7d7-8dd6-43b9-8f75-ad3391d7cb14 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:59.456614 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33d7d7-8dd6-43b9-8f75-ad3391d7cb14 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:59.462649 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:59.463264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.463549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.463873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.464513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:59.466805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:59.552795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1181/2326] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:06:59 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:06:59.556877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:59.567189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7147b15-9674-4b4c-9278-c706707ee1c1 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:59.642386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:59.645158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.645158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.645158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.645158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:59.646105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:59.723869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1146/2327] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:06:59 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:06:59.728835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:59.741270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97dc58-e7a0-4826-a70c-d95c761a2406 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:59.746072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:59.746732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.747012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.747286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.747932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:59.749530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:59.761217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1182/2328] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:06:59.765441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:59.778378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfb0e3-d082-480b-842e-aaff3684f575 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:59.785384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0efbc-2b75-4eac-8d35-617e3a4cf26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:59.786338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0efbc-2b75-4eac-8d35-617e3a4cf26c admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.788582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0efbc-2b75-4eac-8d35-617e3a4cf26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.789097 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0efbc-2b75-4eac-8d35-617e3a4cf26c admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.790245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0efbc-2b75-4eac-8d35-617e3a4cf26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:59.792087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0efbc-2b75-4eac-8d35-617e3a4cf26c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:59.813450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1147/2329] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:06:59.817624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:06:59.828567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01cc87-18d9-43bc-ad56-8c7557b85db5 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:06:59.833287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:06:59.834847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.835141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.835414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] PATH_INFO: `/v3/projects/2fa4a05673a14d1195ca2066b57a6cb8/users/270a3d73e04941a4b410b3e22043db9b/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.851983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa4a05673a14d1195ca2066b57a6cb8, user_id=270a3d73e04941a4b410b3e22043db9b, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:06:59.861079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:06:59.913268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1183/2330] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:06:59 2022] PUT /identity/v3/projects/2fa4a05673a14d1195ca2066b57a6cb8/users/270a3d73e04941a4b410b3e22043db9b/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:06:59.918563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a588-1993-4ed9-a233-d0195540d50c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:06:59.918959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a588-1993-4ed9-a233-d0195540d50c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:06:59.919256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a588-1993-4ed9-a233-d0195540d50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:06:59.938922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-8ac5a588-1993-4ed9-a233-d0195540d50c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:06:59.940273 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8ac5a588-1993-4ed9-a233-d0195540d50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:06:59.997001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:00.032960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1148/2331] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:00.040723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:00.042555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:00.125167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29965b9b-f876-4509-9e79-ecb2e339f771 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:00.140109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:00.140743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:00.141050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:00.141319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:00.152573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:00.154014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:00.175264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1184/2332] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:07:01.427824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa2d3d-4d04-4efb-b047-dc9cccf566fc None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:01.428399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa2d3d-4d04-4efb-b047-dc9cccf566fc None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:01.428639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa2d3d-4d04-4efb-b047-dc9cccf566fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:01.468888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-62fa2d3d-4d04-4efb-b047-dc9cccf566fc None None] MFA Rules not processed for user `270a3d73e04941a4b410b3e22043db9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:01.470281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-62fa2d3d-4d04-4efb-b047-dc9cccf566fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:01.561078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1149/2333] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:01.573433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:01.575003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:01.648131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b5fa-8f21-4c1e-93e1-b39e12fe8504 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:01.651957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:01.652445 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:01.652722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:01.652992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:01.662495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:01.663941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:01.674036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1185/2334] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:07:04.532228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cd41f-1476-4b10-b449-35b1a52d1803 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:04.532930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cd41f-1476-4b10-b449-35b1a52d1803 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:04.532930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cd41f-1476-4b10-b449-35b1a52d1803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:04.555278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-049cd41f-1476-4b10-b449-35b1a52d1803 None None] MFA Rules not processed for user `270a3d73e04941a4b410b3e22043db9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:04.556948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-049cd41f-1476-4b10-b449-35b1a52d1803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:04.564467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1150/2335] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:11.353609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:11.356556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:11.371660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb6f02-3c9a-4ade-bfa0-ad7914f4b11c service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:11.376530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:11.377166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:11.377450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:11.377725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] PATH_INFO: `/v3/users/821cb6d94f7f4b119724c799f8f4a089` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:11.378973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821cb6d94f7f4b119724c799f8f4a089)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:11.381015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:11.393656 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:11.598891 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821cb6d94f7f4b119724c799f8f4a089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:11.599497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821cb6d94f7f4b119724c799f8f4a089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:11.600109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821cb6d94f7f4b119724c799f8f4a089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:11.607351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821cb6d94f7f4b119724c799f8f4a089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:11.688268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821cb6d94f7f4b119724c799f8f4a089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:11.706824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1186/2336] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:07:11 2022] DELETE /identity/v3/users/821cb6d94f7f4b119724c799f8f4a089 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:07:11.711927 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:11.712431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:11.712554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:11.731964 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:11.737029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:11.801656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1151/2337] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:11.808645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:11.811130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:11.879095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d3342-73e1-495e-afea-db468cde6ed5 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:11.882966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc27ea7-84da-4a73-8bae-aa2452c3a3c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:11.883574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27ea7-84da-4a73-8bae-aa2452c3a3c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:11.883851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27ea7-84da-4a73-8bae-aa2452c3a3c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:11.884119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27ea7-84da-4a73-8bae-aa2452c3a3c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:11.894198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc27ea7-84da-4a73-8bae-aa2452c3a3c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:11.896387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc27ea7-84da-4a73-8bae-aa2452c3a3c8 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:11.909863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1187/2338] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:07:12.369597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:12.371095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:12.384274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6577544-3e39-4ae5-aab8-e7f591fe809e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:12.389417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:12.393066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:12.393066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:12.393066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] PATH_INFO: `/v3/projects/0ea4fa264435458cb3d42a5d050bbaa9` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:12.393066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea4fa264435458cb3d42a5d050bbaa9)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:12.394142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:12.530610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea4fa264435458cb3d42a5d050bbaa9', 'request_id': 'req-280bccdd-5335-4f43-a44d-402ba07b1744'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:12.534626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:12.537289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea4fa264435458cb3d42a5d050bbaa9', 'request_id': 'req-280bccdd-5335-4f43-a44d-402ba07b1744'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:12.543484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea4fa264435458cb3d42a5d050bbaa9', 'request_id': 'req-280bccdd-5335-4f43-a44d-402ba07b1744'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:12.620270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] The token cache is being invalidate because project 0ea4fa264435458cb3d42a5d050bbaa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:07:12.620559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:12.621189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-280bccdd-5335-4f43-a44d-402ba07b1744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:12.622806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1152/2339] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:07:12 2022] DELETE /identity/v3/projects/0ea4fa264435458cb3d42a5d050bbaa9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:07:12.627788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ffd9-a197-4f32-baf9-b189ec6f89af None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:12.628337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ffd9-a197-4f32-baf9-b189ec6f89af None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:12.628439 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ffd9-a197-4f32-baf9-b189ec6f89af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:12.648992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8fc0ffd9-a197-4f32-baf9-b189ec6f89af None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:12.651351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8fc0ffd9-a197-4f32-baf9-b189ec6f89af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:12.707676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1188/2340] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:07:12.715517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331fb7d-c636-4fde-a120-1e4473bf0d12 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:12.716004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331fb7d-c636-4fde-a120-1e4473bf0d12 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:12.716108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331fb7d-c636-4fde-a120-1e4473bf0d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:12.737461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3331fb7d-c636-4fde-a120-1e4473bf0d12 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:12.745977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3331fb7d-c636-4fde-a120-1e4473bf0d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:12.749363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1153/2341] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:12.754042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ffd9-a197-4f32-baf9-b189ec6f89af None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:12.764965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ffd9-a197-4f32-baf9-b189ec6f89af None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:12.769812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:12.772139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:12.772139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:12.772139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:12.772139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:07:12.772657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:12.773766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:12.789346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1189/2342] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:07:12.800692 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331fb7d-c636-4fde-a120-1e4473bf0d12 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:12.812154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331fb7d-c636-4fde-a120-1e4473bf0d12 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:12.818178 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:12.818972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:12.819259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:12.819537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:12.820696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:12.822101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:12.922269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1154/2343] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:07:12 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:07:12.923456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:12.935087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6b343-18bf-4fed-a86c-55c5083580f7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:12.952830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:12.996388 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:12.997018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:12.997302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:12.997575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:12.998207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:13.000207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:13.089846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1190/2344] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:07:12 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:07:13.094156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:13.106485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae7133-93ee-4311-a509-907ec03546c3 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:13.111463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:13.112182 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:13.112675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:13.113127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:13.114480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:13.116153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:13.130688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1155/2345] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:07:13.134538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:13.145100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ea861-3b6b-4a82-8014-79bbe1325c0f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:13.150084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f444af-286d-45ad-a9a2-9c7a881fb8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:13.150780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f444af-286d-45ad-a9a2-9c7a881fb8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:13.151060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f444af-286d-45ad-a9a2-9c7a881fb8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:13.151333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f444af-286d-45ad-a9a2-9c7a881fb8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:13.151976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f444af-286d-45ad-a9a2-9c7a881fb8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:13.153645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f444af-286d-45ad-a9a2-9c7a881fb8fb admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:13.171448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1191/2346] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:07:13.172591 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:13.183496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3858da-1f9c-465d-b99b-03dadad23a2b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:13.188441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:13.189187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:13.189532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:13.191367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] PATH_INFO: `/v3/projects/12a1d86b13264fb8901b6ff1eb3e6ee8/users/3894165147ba4d5587021f2ca427fff3/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:13.208521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a1d86b13264fb8901b6ff1eb3e6ee8, user_id=3894165147ba4d5587021f2ca427fff3, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:13.211676 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:13.272172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1156/2347] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:07:13 2022] PUT /identity/v3/projects/12a1d86b13264fb8901b6ff1eb3e6ee8/users/3894165147ba4d5587021f2ca427fff3/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:07:13.283547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b512e64-fe06-40b4-a0bf-446a1e6a354f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:13.283833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b512e64-fe06-40b4-a0bf-446a1e6a354f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:13.284160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b512e64-fe06-40b4-a0bf-446a1e6a354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:13.313976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8b512e64-fe06-40b4-a0bf-446a1e6a354f None None] MFA Rules not processed for user `3894165147ba4d5587021f2ca427fff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:13.315318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8b512e64-fe06-40b4-a0bf-446a1e6a354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:13.389908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1192/2348] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:07:13.399262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:13.400999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:13.458337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3184ec-f61a-4fe9-a1e1-b57cdfa4437a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:13.461831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8a2b5-cc37-4254-ba7d-b266d45f0fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:13.462549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8a2b5-cc37-4254-ba7d-b266d45f0fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:13.462833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8a2b5-cc37-4254-ba7d-b266d45f0fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:13.463102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8a2b5-cc37-4254-ba7d-b266d45f0fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:13.471510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8a2b5-cc37-4254-ba7d-b266d45f0fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:13.472946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8a2b5-cc37-4254-ba7d-b266d45f0fb3 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:13.482643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1157/2349] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:07:13.507294 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b512e64-fe06-40b4-a0bf-446a1e6a354f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:13.571675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b512e64-fe06-40b4-a0bf-446a1e6a354f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:13.576329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:13.576992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:13.577274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:13.577549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] PATH_INFO: `/v3/users/3894165147ba4d5587021f2ca427fff3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:13.578751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3894165147ba4d5587021f2ca427fff3)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:13.580690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:13.751421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:13.765152 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3894165147ba4d5587021f2ca427fff3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:13.765812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3894165147ba4d5587021f2ca427fff3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:13.766485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3894165147ba4d5587021f2ca427fff3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:13.773809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3894165147ba4d5587021f2ca427fff3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:13.865225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3894165147ba4d5587021f2ca427fff3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:13.883822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1193/2350] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:07:13 2022] DELETE /identity/v3/users/3894165147ba4d5587021f2ca427fff3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:07:13.888363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:13.888837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:13.888932 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:13.908318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:13.909544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:13.967091 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1158/2351] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:13.977057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:13.978571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:14.036805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88a9ba-1d0d-4149-9a31-be35e396388d admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:14.040319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4bf0c-3c8e-40ec-be37-ca390d39bd29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:14.040959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4bf0c-3c8e-40ec-be37-ca390d39bd29 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:14.041240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4bf0c-3c8e-40ec-be37-ca390d39bd29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:14.041513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4bf0c-3c8e-40ec-be37-ca390d39bd29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:14.050337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4bf0c-3c8e-40ec-be37-ca390d39bd29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:14.051789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4bf0c-3c8e-40ec-be37-ca390d39bd29 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:14.061318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1194/2352] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:07:14.438643 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:14.450417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:14.457944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d523b6-20f9-476f-b832-bf210f1801b7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:14.464015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:14.464672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:14.464951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:14.465368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] PATH_INFO: `/v3/projects/12a1d86b13264fb8901b6ff1eb3e6ee8` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:14.466628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a1d86b13264fb8901b6ff1eb3e6ee8)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:14.468387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:14.490427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:14.575571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a1d86b13264fb8901b6ff1eb3e6ee8', 'request_id': 'req-4ab27d20-59d1-4441-8cd5-7c2f19777718'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:14.580960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a1d86b13264fb8901b6ff1eb3e6ee8', 'request_id': 'req-4ab27d20-59d1-4441-8cd5-7c2f19777718'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:14.587881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a1d86b13264fb8901b6ff1eb3e6ee8', 'request_id': 'req-4ab27d20-59d1-4441-8cd5-7c2f19777718'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:14.659621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] The token cache is being invalidate because project 12a1d86b13264fb8901b6ff1eb3e6ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:07:14.659938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:14.660593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4ab27d20-59d1-4441-8cd5-7c2f19777718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:07:14.662298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1159/2353] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:07:14 2022] DELETE /identity/v3/projects/12a1d86b13264fb8901b6ff1eb3e6ee8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:07:14.667491 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c30a7-be85-4a69-944e-ac136e50dc3e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:14.667782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c30a7-be85-4a69-944e-ac136e50dc3e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:14.668064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c30a7-be85-4a69-944e-ac136e50dc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:14.687525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-864c30a7-be85-4a69-944e-ac136e50dc3e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:14.688731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-864c30a7-be85-4a69-944e-ac136e50dc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:14.747579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1195/2354] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:14.756063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c169b-142f-4573-baa1-9eab665a1fcc None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:14.756390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c169b-142f-4573-baa1-9eab665a1fcc None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:14.756667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c169b-142f-4573-baa1-9eab665a1fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:14.777507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-901c169b-142f-4573-baa1-9eab665a1fcc None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:14.778736 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-901c169b-142f-4573-baa1-9eab665a1fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:14.786888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1160/2355] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:07:14.791133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c30a7-be85-4a69-944e-ac136e50dc3e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:14.800897 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c30a7-be85-4a69-944e-ac136e50dc3e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:14.805626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:14.806190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:14.806508 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:14.806782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:14.807424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:07:14.807725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:14.809327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:14.821454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1196/2356] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:07:14.825580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c169b-142f-4573-baa1-9eab665a1fcc None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:14.834972 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c169b-142f-4573-baa1-9eab665a1fcc None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:14.839594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:14.840219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:14.840560 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:14.840838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:14.841455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:14.843377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:14.923605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1161/2357] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:07:14 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:07:14.928324 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:14.938489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f6dd8-0af9-4bfd-b512-80c62ab1ee31 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:14.972759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:14.996824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:14.997424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:14.997700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:14.997974 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:14.998703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:15.000696 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:15.122310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1197/2358] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:07:14 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:07:15.132695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:15.142937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fe75f-1bee-4bb4-b0ee-23d1471afc23 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:15.147454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:15.148093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:15.148370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:15.148644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:15.149366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:15.151000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:15.160779 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1162/2359] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:07:15.164678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:15.174666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b6379-e8d7-47b6-b629-568967bb652c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:15.179928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a73497-dd94-4f0f-9d86-cb2a09145e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:15.180514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a73497-dd94-4f0f-9d86-cb2a09145e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:15.180792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a73497-dd94-4f0f-9d86-cb2a09145e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:15.181064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a73497-dd94-4f0f-9d86-cb2a09145e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:15.181699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a73497-dd94-4f0f-9d86-cb2a09145e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:15.183314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a73497-dd94-4f0f-9d86-cb2a09145e82 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:15.193576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1198/2360] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:07:15.197372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:15.207031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84638af-a9f1-4900-9f4d-2528e9064b3b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:15.211599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:15.212232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:15.212508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:15.212783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] PATH_INFO: `/v3/projects/0533d69c876f4cbdbf6b7138abf37893/users/93ec72fc30ea45d5952bf9f29311e4ac/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:15.227276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0533d69c876f4cbdbf6b7138abf37893, user_id=93ec72fc30ea45d5952bf9f29311e4ac, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:15.229883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:15.298724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1163/2361] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:07:15 2022] PUT /identity/v3/projects/0533d69c876f4cbdbf6b7138abf37893/users/93ec72fc30ea45d5952bf9f29311e4ac/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:07:15.303892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:15.304186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:15.304500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:15.324129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:15.325352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:15.382969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1199/2362] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:15.392367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:15.394037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:15.449594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89029a-7d78-44e8-a0d7-ebc7958a7d1c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:15.453296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0e9ff-80df-48a3-8edd-77947db4d716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:15.453894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0e9ff-80df-48a3-8edd-77947db4d716 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:15.454171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0e9ff-80df-48a3-8edd-77947db4d716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:15.454468 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0e9ff-80df-48a3-8edd-77947db4d716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:15.465509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0e9ff-80df-48a3-8edd-77947db4d716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:15.467188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0e9ff-80df-48a3-8edd-77947db4d716 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:15.477340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1164/2363] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:07:20.841055 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:20.843103 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:21.032812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12a1d1-3f46-4e46-8d3d-3509145470f4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:21.038795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:21.040349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.040953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.041475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.043562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:21.062872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:21.067247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:21.083391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1200/2364] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:07:21.229103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea6155-9a64-4a18-b6e8-564c006b2531 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.229639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea6155-9a64-4a18-b6e8-564c006b2531 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.229722 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea6155-9a64-4a18-b6e8-564c006b2531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.261247 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d2ea6155-9a64-4a18-b6e8-564c006b2531 None None] MFA Rules not processed for user `93ec72fc30ea45d5952bf9f29311e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:21.262531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d2ea6155-9a64-4a18-b6e8-564c006b2531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:21.296580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:21.335240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1165/2365] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:21.337959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:21.364364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae967309-80fe-4290-9d73-e5df5d89011e service placement] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:21.369440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:21.370052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.370391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.370670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.371303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:21.373197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:21.490483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1201/2366] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:07:21 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:07:21.494661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea6155-9a64-4a18-b6e8-564c006b2531 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:21.506613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea6155-9a64-4a18-b6e8-564c006b2531 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:21.574150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:21.574865 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.575149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.575445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.576088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:21.578061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:21.656327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1166/2367] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:07:21 2022] POST /identity/v3/users => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:07:21.661546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:21.674191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d20ba-7933-4ce4-99d8-820d971ce811 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:21.679543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:21.680157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.680503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.680790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.681371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:21.683612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:21.695620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1202/2368] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:07:21.699351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:21.711537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02b685-6e7e-4bbb-839c-2949e1136e8c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:21.716929 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:21.717613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.717893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.718165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] PATH_INFO: `/v3/projects/78ef109fc3a24910b73a08434559a8f6/users/9894bf1d999e4739ab1aec2ffd4387bd/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.736518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ef109fc3a24910b73a08434559a8f6, user_id=9894bf1d999e4739ab1aec2ffd4387bd, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:21.740491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:21.808898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1167/2369] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:07:21 2022] PUT /identity/v3/projects/78ef109fc3a24910b73a08434559a8f6/users/9894bf1d999e4739ab1aec2ffd4387bd/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:07:21.813122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:21.836121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952021a-f192-4a30-8c6d-ad4623654f0f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:21.846459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:21.903852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac97a8f-753f-42c7-b085-c00cf45eb9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:21.904492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97a8f-753f-42c7-b085-c00cf45eb9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.904779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97a8f-753f-42c7-b085-c00cf45eb9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.905262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac97a8f-753f-42c7-b085-c00cf45eb9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.905921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac97a8f-753f-42c7-b085-c00cf45eb9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:21.907633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac97a8f-753f-42c7-b085-c00cf45eb9b8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:21.920914 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1203/2370] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:07:21.925076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:21.938272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f47b4-4e81-4a96-8e90-2e18f96ad810 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:21.943738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:21.944461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:21.944767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:21.945042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] PATH_INFO: `/v3/projects/78ef109fc3a24910b73a08434559a8f6/users/9894bf1d999e4739ab1aec2ffd4387bd/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:21.947127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ef109fc3a24910b73a08434559a8f6, user_id=9894bf1d999e4739ab1aec2ffd4387bd, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:21.949787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:21.963355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1168/2371] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:07:21 2022] PUT /identity/v3/projects/78ef109fc3a24910b73a08434559a8f6/users/9894bf1d999e4739ab1aec2ffd4387bd/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:07:22.914059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d97b-b6b0-4a1b-b6e7-320c6dfd44df None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:22.914677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d97b-b6b0-4a1b-b6e7-320c6dfd44df None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:22.915043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182d97b-b6b0-4a1b-b6e7-320c6dfd44df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:22.935524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f182d97b-b6b0-4a1b-b6e7-320c6dfd44df None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:22.937271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f182d97b-b6b0-4a1b-b6e7-320c6dfd44df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:22.996730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1204/2372] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 22 11:07:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:23.010020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:23.011894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:23.094937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387479b-60fd-4e39-98ee-3395e80099dc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:23.100537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:23.101046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:23.101445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:23.101589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:23.112314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:23.116688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:23.127863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1169/2373] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:07:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:07:28.397918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da14f-1122-4c13-9c95-b5534936221c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:28.400968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da14f-1122-4c13-9c95-b5534936221c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:28.401412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202da14f-1122-4c13-9c95-b5534936221c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:28.452505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-202da14f-1122-4c13-9c95-b5534936221c None None] MFA Rules not processed for user `9894bf1d999e4739ab1aec2ffd4387bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:28.453672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-202da14f-1122-4c13-9c95-b5534936221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:28.543788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1205/2374] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:28.557418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:28.559373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:28.582084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:28.624810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aad191-fa95-49be-8b99-5c2e2ec9e9a2 service swift] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:28.630170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990660ce5a9f4213a5b42b3824d0b567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:28.631117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:28.631614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:28.631955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:28.698539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:28.700011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:28.710918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1170/2375] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:07:28.992691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3b2ec-72dc-4570-b77b-6bfa4c5f44a8 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:28.992986 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3b2ec-72dc-4570-b77b-6bfa4c5f44a8 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:28.993267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3b2ec-72dc-4570-b77b-6bfa4c5f44a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:29.014752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-31d3b2ec-72dc-4570-b77b-6bfa4c5f44a8 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:29.016074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-31d3b2ec-72dc-4570-b77b-6bfa4c5f44a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:29.027017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1206/2376] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:29.040415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:29.052908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810d3f1-f8f8-4762-ae93-592778c2fe1c service glance] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:29.057956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693e18a-aa61-40b8-bb93-23e1eb9c0fd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:29.058681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e18a-aa61-40b8-bb93-23e1eb9c0fd9 service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:29.064552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e18a-aa61-40b8-bb93-23e1eb9c0fd9 service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:29.064552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693e18a-aa61-40b8-bb93-23e1eb9c0fd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:29.070513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e18a-aa61-40b8-bb93-23e1eb9c0fd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:29.072140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693e18a-aa61-40b8-bb93-23e1eb9c0fd9 service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:29.081777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1171/2377] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:07:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:07:39.009971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=85 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:39.077510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=47 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:41.272700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1302b7-ff84-42b2-bb5f-26168ffc2a6e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:41.273184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1302b7-ff84-42b2-bb5f-26168ffc2a6e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:41.273287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1302b7-ff84-42b2-bb5f-26168ffc2a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:41.291865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-ba1302b7-ff84-42b2-bb5f-26168ffc2a6e None None] MFA Rules not processed for user `93ec72fc30ea45d5952bf9f29311e4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:41.294427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ba1302b7-ff84-42b2-bb5f-26168ffc2a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:41.300435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1207/2378] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:07:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:45.983787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1928c8-ff9c-48d2-92a6-d550ad856bf4 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:45.984335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1928c8-ff9c-48d2-92a6-d550ad856bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:45.984335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1928c8-ff9c-48d2-92a6-d550ad856bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:46.017677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-fa1928c8-ff9c-48d2-92a6-d550ad856bf4 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:07:46.019276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fa1928c8-ff9c-48d2-92a6-d550ad856bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:07:46.034999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1172/2379] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:07:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:07:46.043413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1302b7-ff84-42b2-bb5f-26168ffc2a6e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:07:46.064239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1302b7-ff84-42b2-bb5f-26168ffc2a6e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:07:46.073894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d8813-94fb-4e99-9274-224ced8d187d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:07:46.078640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8813-94fb-4e99-9274-224ced8d187d service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:07:46.079555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8813-94fb-4e99-9274-224ced8d187d service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:07:46.080603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d8813-94fb-4e99-9274-224ced8d187d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:07:46.097633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d8813-94fb-4e99-9274-224ced8d187d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:07:46.100778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d8813-94fb-4e99-9274-224ced8d187d service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:07:46.116693 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1208/2380] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:07:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:07:56.011897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:07:56.111979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=9 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:08:00.248816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f48-626f-4b5a-aec3-1e415ae1f09a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:08:00.249308 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f48-626f-4b5a-aec3-1e415ae1f09a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:08:00.249419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4f48-626f-4b5a-aec3-1e415ae1f09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:08:00.268334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5dee4f48-626f-4b5a-aec3-1e415ae1f09a None None] MFA Rules not processed for user `a1cdc3bfa3e046fa863131d7ff27d4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:08:00.269604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5dee4f48-626f-4b5a-aec3-1e415ae1f09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:08:00.323161 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1173/2381] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:08:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:08:00.792911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:08:00.793501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:08:00.793501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:08:00.817675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] MFA Rules not processed for user `7c6fee0b956e41a7b2d9b914043633ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:08:00.818936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:08:00.888943 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1209/2382] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:08:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:08:10.316636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=25 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:08:10.881256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=25 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:08:28.834785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944deab-1593-4ad2-bf91-27a1149b7906 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:08:28.835361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944deab-1593-4ad2-bf91-27a1149b7906 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:08:28.835478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944deab-1593-4ad2-bf91-27a1149b7906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:08:28.854879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e944deab-1593-4ad2-bf91-27a1149b7906 None None] MFA Rules not processed for user `d13b7df898df4d238ea290fab8edc4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:08:28.857108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e944deab-1593-4ad2-bf91-27a1149b7906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:08:28.922088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1174/2383] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:08:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:08:30.413499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:08:30.414960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:08:30.471438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a962c2c-2ccf-49d4-89ab-4210c23d77e4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:08:30.475049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:08:30.475650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:08:30.475941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:08:30.476216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:08:30.485521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:08:30.486989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:08:30.497015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1210/2384] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:08:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:08:35.407815 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dff686-1b71-4bb4-96c9-fe446af3c850 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:08:35.409202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dff686-1b71-4bb4-96c9-fe446af3c850 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:08:35.409483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dff686-1b71-4bb4-96c9-fe446af3c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:08:35.430634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-54dff686-1b71-4bb4-96c9-fe446af3c850 None None] MFA Rules not processed for user `270a3d73e04941a4b410b3e22043db9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:08:35.431651 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-54dff686-1b71-4bb4-96c9-fe446af3c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:08:35.507185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1175/2385] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:08:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:08:40.492365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=26 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:08:45.500219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=50 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:13.796824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:13.806021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e3588-09d1-436e-a2b7-8754593ed9f6 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:13.811250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:13.811250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:13.811550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:13.811653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:13.813355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:13.818275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dff686-1b71-4bb4-96c9-fe446af3c850 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:13.829908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dff686-1b71-4bb4-96c9-fe446af3c850 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:13.833721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:13.835354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:13.835354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:13.835354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:13.836915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:13.896960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:13.898625 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:13.910310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1211/2386] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:09:13.913151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:13.914818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:13.926561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1176/2387] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:09:23.906256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=34 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:23.922272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=34 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:25.305494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:25.307128 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:25.322446 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ccd2f-0be0-439a-bfe9-2ec26ddea852 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:25.328290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:25.328908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:25.329798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:25.330081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:25.331938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:25.344949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:25.346998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:25.357726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1212/2388] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:28.631483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:28.632951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:28.689460 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c5c3-0166-4319-99b1-9be543d3345e service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:28.693032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:28.693633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:28.693913 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:28.694187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:28.695897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:28.706786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:28.708207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:28.717108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1177/2389] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:35.153850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:35.164233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3ff3-5da8-4293-adf7-3749b0754ab3 service nova] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:35.168094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:35.168748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:35.169028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:35.169298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:35.171220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:35.237393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:35.238928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:35.256495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1213/2390] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:38.713093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=26 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:39.399538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:39.402042 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:39.418824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73e3c5-f038-4f56-acdd-b62143788ad0 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:39.423979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:39.428201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:39.428201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:39.428201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] PATH_INFO: `/v3/users/7c6fee0b956e41a7b2d9b914043633ea` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:39.428201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6fee0b956e41a7b2d9b914043633ea)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:39.429071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:39.640851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6fee0b956e41a7b2d9b914043633ea'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:39.641605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6fee0b956e41a7b2d9b914043633ea'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:39.642383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6fee0b956e41a7b2d9b914043633ea'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:39.741526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6fee0b956e41a7b2d9b914043633ea'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:39.750567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-1ac046f2-9a97-4099-8ca0-b28b8bf074b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6fee0b956e41a7b2d9b914043633ea'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:39.778026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1178/2391] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:09:39 2022] DELETE /identity/v3/users/7c6fee0b956e41a7b2d9b914043633ea => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:40.081668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:40.105021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43301a98-c017-4b6a-b7bd-c8e6ff655deb service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:40.197391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:40.198131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.198506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.198987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] PATH_INFO: `/v3/projects/2cb7651353884366b2ca72a7e272bdc0` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.200431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb7651353884366b2ca72a7e272bdc0)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:40.207369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:40.325059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb7651353884366b2ca72a7e272bdc0', 'request_id': 'req-53575106-2228-4501-9dbc-2fdeb7f7a081'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:40.330622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb7651353884366b2ca72a7e272bdc0', 'request_id': 'req-53575106-2228-4501-9dbc-2fdeb7f7a081'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:40.337219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb7651353884366b2ca72a7e272bdc0', 'request_id': 'req-53575106-2228-4501-9dbc-2fdeb7f7a081'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:40.421532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] The token cache is being invalidate because project 2cb7651353884366b2ca72a7e272bdc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:09:40.421909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:40.422634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-53575106-2228-4501-9dbc-2fdeb7f7a081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:40.424684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1214/2392] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:09:40 2022] DELETE /identity/v3/projects/2cb7651353884366b2ca72a7e272bdc0 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:40.429657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915449ac-0cdd-41ad-8ecc-9691a1a3e531 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.429963 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915449ac-0cdd-41ad-8ecc-9691a1a3e531 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.430263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915449ac-0cdd-41ad-8ecc-9691a1a3e531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.450941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-915449ac-0cdd-41ad-8ecc-9691a1a3e531 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:40.452130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-915449ac-0cdd-41ad-8ecc-9691a1a3e531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:40.513017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1179/2393] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:40.521475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084237-e42f-4f5f-840c-0179bc036531 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.521874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084237-e42f-4f5f-840c-0179bc036531 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.522223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084237-e42f-4f5f-840c-0179bc036531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.543327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d4084237-e42f-4f5f-840c-0179bc036531 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:40.544553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d4084237-e42f-4f5f-840c-0179bc036531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:40.553308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1215/2394] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:40.557799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915449ac-0cdd-41ad-8ecc-9691a1a3e531 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:40.567976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915449ac-0cdd-41ad-8ecc-9691a1a3e531 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:40.572590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:40.574863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.574863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.574863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.574863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:09:40.574863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:40.576352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:40.588688 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1180/2395] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:09:40.592980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4084237-e42f-4f5f-840c-0179bc036531 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:40.599621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:40.606319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4084237-e42f-4f5f-840c-0179bc036531 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:40.608456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:40.609049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.609327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.609597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.610220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:40.612549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:40.731626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1216/2396] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:09:40 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:09:40.735845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:40.746347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb2ef7-76b9-410c-acc1-cddb20e9723e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:40.805810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:40.806446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.806726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.806997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.807612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:40.809993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:40.818141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:40.890525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1181/2397] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:09:40 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:09:40.895248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:40.905553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b511f93-4bb0-4ea0-8c4f-4eba96ec6297 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:40.910186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:40.910823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.911121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.911647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.912050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:40.913634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:40.926581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1217/2398] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:09:40.930734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:40.941417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0bf4cb-3416-4880-9aee-ce2ccc652c6e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:40.946085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c3e1c-a919-43c0-82ed-12a93c594867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:40.956033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3e1c-a919-43c0-82ed-12a93c594867 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.956033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3e1c-a919-43c0-82ed-12a93c594867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.956033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c3e1c-a919-43c0-82ed-12a93c594867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.956033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c3e1c-a919-43c0-82ed-12a93c594867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:40.956033 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c3e1c-a919-43c0-82ed-12a93c594867 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:40.961071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1182/2399] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:09:40.965305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:40.975550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82393eb-9b42-4b43-8fb3-0b472f811006 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:40.980174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:40.980886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:40.984710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:40.984710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] PATH_INFO: `/v3/projects/90890097847d42d5aa6090b728b4f549/users/f3e830aed9b84b0ebbc4e2ddab25921e/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:40.998253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90890097847d42d5aa6090b728b4f549, user_id=f3e830aed9b84b0ebbc4e2ddab25921e, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:41.001701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:41.092193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1218/2400] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:09:40 2022] PUT /identity/v3/projects/90890097847d42d5aa6090b728b4f549/users/f3e830aed9b84b0ebbc4e2ddab25921e/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:41.103296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:41.103582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:41.103854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:41.136726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] MFA Rules not processed for user `f3e830aed9b84b0ebbc4e2ddab25921e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:41.139102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:41.250732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1183/2401] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:41.262076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:41.265790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:41.357052 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419c189-1783-4036-af8f-557f97befa82 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:41.362551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:41.363187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:41.363466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:41.363741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:41.377722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:41.379216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:41.392003 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1219/2402] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:42.944775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:42.946255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:43.028659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ee53b-8edb-4954-adb5-a11e12af1fca None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:43.034890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0d33e-ced3-4ec3-8fc3-dc94fe94a5c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:43.035512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0d33e-ced3-4ec3-8fc3-dc94fe94a5c9 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:43.035793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0d33e-ced3-4ec3-8fc3-dc94fe94a5c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:43.036068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0d33e-ced3-4ec3-8fc3-dc94fe94a5c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:43.037912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b4f0d33e-ced3-4ec3-8fc3-dc94fe94a5c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:43.098004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0d33e-ced3-4ec3-8fc3-dc94fe94a5c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:43.099724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0d33e-ced3-4ec3-8fc3-dc94fe94a5c9 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:43.108889 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1184/2403] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:44.700860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:44.703455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:44.775903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0d161-48e8-474a-ad59-c0557f51aa9e service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:44.780839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:44.781670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:44.781947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:44.782220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] PATH_INFO: `/v3/users/270a3d73e04941a4b410b3e22043db9b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:44.783533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=270a3d73e04941a4b410b3e22043db9b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:44.785502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:44.950053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '270a3d73e04941a4b410b3e22043db9b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:44.950755 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '270a3d73e04941a4b410b3e22043db9b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:44.951350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '270a3d73e04941a4b410b3e22043db9b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:44.955935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:44.958839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '270a3d73e04941a4b410b3e22043db9b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:45.035625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:45.036223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:45.036381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:45.047234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:45.058959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] MFA Rules not processed for user `a1cdc3bfa3e046fa863131d7ff27d4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:45.060208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:45.068341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '270a3d73e04941a4b410b3e22043db9b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:45.087207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1220/2404] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:09:44 2022] DELETE /identity/v3/users/270a3d73e04941a4b410b3e22043db9b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:45.131785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1185/2405] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:45.439478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:45.440994 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:45.524489 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bbdac-91b5-4b68-80af-5e868dff34ac admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:45.529711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:45.530498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:45.530968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:45.531339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] PATH_INFO: `/v3/projects/2fa4a05673a14d1195ca2066b57a6cb8` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:45.533294 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa4a05673a14d1195ca2066b57a6cb8)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:45.535466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:45.537886 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:45.538860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:45.601743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83948d5d-f238-4e47-9430-ed17dab7ee8b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:45.651028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa4a05673a14d1195ca2066b57a6cb8', 'request_id': 'req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:45.653326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f719e-c300-44f7-9587-708e08ffbfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:45.653944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f719e-c300-44f7-9587-708e08ffbfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:45.654225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f719e-c300-44f7-9587-708e08ffbfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:45.654527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f719e-c300-44f7-9587-708e08ffbfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:45.656482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b92f719e-c300-44f7-9587-708e08ffbfe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:45.656655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa4a05673a14d1195ca2066b57a6cb8', 'request_id': 'req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:45.663422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa4a05673a14d1195ca2066b57a6cb8', 'request_id': 'req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:45.666860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f719e-c300-44f7-9587-708e08ffbfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:45.668319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f719e-c300-44f7-9587-708e08ffbfe4 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:45.676553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1186/2406] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:09:45.741127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] The token cache is being invalidate because project 2fa4a05673a14d1195ca2066b57a6cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:09:45.741754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:45.743370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7bac665d-f9b3-4b9a-a1c1-17e28a0ee02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:45.746832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1221/2407] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:09:45 2022] DELETE /identity/v3/projects/2fa4a05673a14d1195ca2066b57a6cb8 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:45.753445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a46bc-bf8a-43f0-a2b0-f3d669ece7d0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:45.753778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a46bc-bf8a-43f0-a2b0-f3d669ece7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:45.754121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a46bc-bf8a-43f0-a2b0-f3d669ece7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:45.777314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e01a46bc-bf8a-43f0-a2b0-f3d669ece7d0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:45.778574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e01a46bc-bf8a-43f0-a2b0-f3d669ece7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:45.897616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1187/2408] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:45.902926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57da0e1-d5b9-4867-8bf8-df9785d16a61 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:45.903215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57da0e1-d5b9-4867-8bf8-df9785d16a61 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:45.903492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57da0e1-d5b9-4867-8bf8-df9785d16a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:45.924488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c57da0e1-d5b9-4867-8bf8-df9785d16a61 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:45.925688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c57da0e1-d5b9-4867-8bf8-df9785d16a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:45.934474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1222/2409] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:09:45.944145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a46bc-bf8a-43f0-a2b0-f3d669ece7d0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:45.953862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a46bc-bf8a-43f0-a2b0-f3d669ece7d0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:45.959387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:45.960009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:45.960292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:45.960644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:45.961300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:09:45.961598 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:45.963266 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:45.967017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:45.992544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1188/2410] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:09:46.000126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57da0e1-d5b9-4867-8bf8-df9785d16a61 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:46.012679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57da0e1-d5b9-4867-8bf8-df9785d16a61 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:46.019164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:46.019789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.020274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.020542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.021319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:46.023259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:46.142323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:46.142323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:46.182556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1223/2411] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:09:45 2022] POST /identity/v3/projects => generated 389 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:09:46.187146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:46.232362 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:46.238600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11858e0e-150c-4136-b3e5-46d3e2f7ff29 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:46.261540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798ede1-2cb0-48da-b652-6f0bf629b43a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:46.266440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:46.267079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.267360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.267632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.268253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:46.270167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:46.300320 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:46.300961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.301245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.301527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.303288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:46.314656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:46.316259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:46.328264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1189/2412] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:09:46.376913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1224/2413] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:09:46 2022] POST /identity/v3/users => generated 295 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:09:46.381143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:46.391510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636d6e7-2b76-41a0-9bc9-c0feccc9c310 service placement] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:46.397537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:46.398289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.398639 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.399002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.399684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:46.401404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:46.413053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1190/2414] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:09:46.417200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:46.427544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4cccc-01d2-4e71-a6d9-f94eb69fac7a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:46.432248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58452a-834c-4d1b-82ef-5dcff08cf501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:46.432917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58452a-834c-4d1b-82ef-5dcff08cf501 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.433260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58452a-834c-4d1b-82ef-5dcff08cf501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.433653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58452a-834c-4d1b-82ef-5dcff08cf501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.434624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58452a-834c-4d1b-82ef-5dcff08cf501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:46.436631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58452a-834c-4d1b-82ef-5dcff08cf501 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:46.451022 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1225/2415] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:09:46.452153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:46.463485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636e4b6-f55c-4f27-a0cd-73c0e0458e4c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:46.468518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:46.469260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.469644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.470029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] PATH_INFO: `/v3/projects/496d4105ca2e443bb4fcbd76e8ce0537/users/4be79b35cb3141689838d2b637ddf82c/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.494593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496d4105ca2e443bb4fcbd76e8ce0537, user_id=4be79b35cb3141689838d2b637ddf82c, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:46.494593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:46.564356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1191/2416] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:09:46 2022] PUT /identity/v3/projects/496d4105ca2e443bb4fcbd76e8ce0537/users/4be79b35cb3141689838d2b637ddf82c/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:46.574766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.575057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.575331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.604214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] MFA Rules not processed for user `4be79b35cb3141689838d2b637ddf82c`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:46.605487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:46.672893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1226/2417] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:46.684012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:46.685677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:46.745208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2373fa4-9127-43c8-a0b5-3760de87d9a7 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:46.749104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:46.749592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:46.749870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:46.750142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:46.757885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:46.760508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:46.763305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:46.771713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1192/2418] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:52.309703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:52.311239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:52.367810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c439e-e9dc-4445-aaf3-d152c00bcd44 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:52.371483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:52.372088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:52.372359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:52.372626 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:52.374367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:52.384605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:52.386025 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:52.393920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1227/2419] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:53.083810 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:53.085567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:53.176250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30d390-180d-4553-95f4-a5dbc53703b0 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:53.182009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947cef41-3ba5-4f23-ab0b-60240d747358 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:53.182666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cef41-3ba5-4f23-ab0b-60240d747358 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:53.182965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cef41-3ba5-4f23-ab0b-60240d747358 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:53.183237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947cef41-3ba5-4f23-ab0b-60240d747358 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:53.185019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-947cef41-3ba5-4f23-ab0b-60240d747358 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:53.200626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cef41-3ba5-4f23-ab0b-60240d747358 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:53.202621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947cef41-3ba5-4f23-ab0b-60240d747358 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:53.218926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1193/2420] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:53.964472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:53.965885 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:54.011271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:54.031062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3fd7e-828d-4958-8f80-9f35bef7da78 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:54.035698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:54.036322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:54.036597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:54.036940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] PATH_INFO: `/v3/users/f3e830aed9b84b0ebbc4e2ddab25921e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:54.038315 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e830aed9b84b0ebbc4e2ddab25921e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:54.040978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:54.206825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e830aed9b84b0ebbc4e2ddab25921e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:54.207483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e830aed9b84b0ebbc4e2ddab25921e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:54.208092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e830aed9b84b0ebbc4e2ddab25921e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:54.214944 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e830aed9b84b0ebbc4e2ddab25921e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:54.267360 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e830aed9b84b0ebbc4e2ddab25921e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:54.286498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1228/2421] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:09:53 2022] DELETE /identity/v3/users/f3e830aed9b84b0ebbc4e2ddab25921e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:54.291159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:54.291667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:54.291853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:54.311801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:54.313016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:54.375429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1194/2422] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:54.385079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:54.386543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:54.446969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce156e-15b2-4a0c-b820-281d31150428 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:54.451345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad213c-b0af-459d-aca8-48712df48c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:54.451956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad213c-b0af-459d-aca8-48712df48c3a service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:54.452234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad213c-b0af-459d-aca8-48712df48c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:54.452507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad213c-b0af-459d-aca8-48712df48c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:54.462788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad213c-b0af-459d-aca8-48712df48c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:54.464238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad213c-b0af-459d-aca8-48712df48c3a service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:54.474184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1229/2423] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:54.983411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:54.984858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:54.997279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a87de-cdee-49e4-9516-27a46c11b77c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:55.002914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:55.003677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.004010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.004248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] PATH_INFO: `/v3/projects/90890097847d42d5aa6090b728b4f549` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.005379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90890097847d42d5aa6090b728b4f549)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:55.007409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:55.113488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90890097847d42d5aa6090b728b4f549', 'request_id': 'req-09429894-55ea-4528-b70f-bb04b9079509'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:55.118982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90890097847d42d5aa6090b728b4f549', 'request_id': 'req-09429894-55ea-4528-b70f-bb04b9079509'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:55.125225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90890097847d42d5aa6090b728b4f549', 'request_id': 'req-09429894-55ea-4528-b70f-bb04b9079509'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:55.249691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] The token cache is being invalidate because project 90890097847d42d5aa6090b728b4f549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:09:55.249908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:55.250551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-09429894-55ea-4528-b70f-bb04b9079509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:09:55.252693 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1195/2424] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:09:54 2022] DELETE /identity/v3/projects/90890097847d42d5aa6090b728b4f549 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:55.257735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5de3b5-51f6-46c2-940a-74a6e717aa26 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.258193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5de3b5-51f6-46c2-940a-74a6e717aa26 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.258312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5de3b5-51f6-46c2-940a-74a6e717aa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.279023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bc5de3b5-51f6-46c2-940a-74a6e717aa26 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:55.280218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bc5de3b5-51f6-46c2-940a-74a6e717aa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:55.338849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1230/2425] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:55.348086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c19c-5ec5-4419-ae77-acc5c5991ec7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.348553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c19c-5ec5-4419-ae77-acc5c5991ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.348648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800c19c-5ec5-4419-ae77-acc5c5991ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.357104 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:55.368748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2800c19c-5ec5-4419-ae77-acc5c5991ec7 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:55.369985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2800c19c-5ec5-4419-ae77-acc5c5991ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:55.378483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1196/2426] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:55.382862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5de3b5-51f6-46c2-940a-74a6e717aa26 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:55.388954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:55.392619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5de3b5-51f6-46c2-940a-74a6e717aa26 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:55.397472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:55.398091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.398413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.398690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.399367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:09:55.399669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:55.401247 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:55.412733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1231/2427] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:09:55.417013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800c19c-5ec5-4419-ae77-acc5c5991ec7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:55.427170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800c19c-5ec5-4419-ae77-acc5c5991ec7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:55.432000 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:55.432632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.434285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.434285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.434285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:55.436246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:55.504502 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1197/2428] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:09:55 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:09:55.507585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:55.518098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a69f3c-9642-46cd-818b-29493d3256dc admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:55.575737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:55.576351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.576631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.576901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.577520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:55.579467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:55.770902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1232/2429] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:09:55 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:09:55.774695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:55.785499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecf8dc-9a8f-4c00-b36b-d78bedf81c0f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:55.790022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:55.790669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.790946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.791216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.791854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:55.793453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:55.806577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1198/2430] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:09:55.812993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:55.823870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a141485-e8af-4bfa-8155-23fcd5df97d6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:55.832067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017662f-8f13-4c82-abdb-1c0caf193731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:55.832800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017662f-8f13-4c82-abdb-1c0caf193731 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.833080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017662f-8f13-4c82-abdb-1c0caf193731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.833354 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017662f-8f13-4c82-abdb-1c0caf193731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.833996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017662f-8f13-4c82-abdb-1c0caf193731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:55.836104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017662f-8f13-4c82-abdb-1c0caf193731 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:55.848061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1233/2431] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:09:55.852205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:55.862836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62617ff-c722-4852-96ea-21f9ea0384f8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:55.867438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:55.868099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:55.868377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:55.868652 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] PATH_INFO: `/v3/projects/291865d4372641af8d60c6cd6e8fc0dc/users/d952d186196846f78d586ebd0353b747/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:55.884118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] RBAC: Authorizing `identity:create_grant(project_id=291865d4372641af8d60c6cd6e8fc0dc, user_id=d952d186196846f78d586ebd0353b747, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:55.886883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.033383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1199/2432] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:09:55 2022] PUT /identity/v3/projects/291865d4372641af8d60c6cd6e8fc0dc/users/d952d186196846f78d586ebd0353b747/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:56.043941 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d214-202f-40e0-885e-e6b979e96e73 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.044227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d214-202f-40e0-885e-e6b979e96e73 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.044505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277d214-202f-40e0-885e-e6b979e96e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.072826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9277d214-202f-40e0-885e-e6b979e96e73 None None] MFA Rules not processed for user `d952d186196846f78d586ebd0353b747`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:56.074044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9277d214-202f-40e0-885e-e6b979e96e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:56.140183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1234/2433] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:09:56.144667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:56.206396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7971b8-eea8-4cc6-891f-d95a53e9e17a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:56.211213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:56.211816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.212123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.212399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.213018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:56.214996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.302377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1200/2434] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:09:56 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:09:56.306564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277d214-202f-40e0-885e-e6b979e96e73 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:56.317225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277d214-202f-40e0-885e-e6b979e96e73 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:56.373855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:56.374488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.374771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.375046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.375665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:56.377712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.385377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:56.484496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1235/2435] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:09:56 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:09:56.488873 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:56.499088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039988e-c955-4d61-979d-3e756d0a184c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:56.504271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:56.504859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.505139 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.505410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.506098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:56.507717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.518924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1201/2436] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:09:56.522791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:56.532711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309f7bd-24e1-4708-ad4b-8e1fec5c218f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:56.537936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:56.538603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.538880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.539153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] PATH_INFO: `/v3/projects/d1fcf58794fc47738ed845189904fcfa/users/36369de0ba02413bb685985412adbf6f/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.553736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1fcf58794fc47738ed845189904fcfa, user_id=36369de0ba02413bb685985412adbf6f, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:56.556329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.604607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1236/2437] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:09:56 2022] PUT /identity/v3/projects/d1fcf58794fc47738ed845189904fcfa/users/36369de0ba02413bb685985412adbf6f/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:09:56.608612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:56.619109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df797d25-4606-47a3-a103-6ee70ee519ba admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:56.679388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0010c5-c776-41b3-9d81-68856bb80316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:56.680562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0010c5-c776-41b3-9d81-68856bb80316 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.680562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0010c5-c776-41b3-9d81-68856bb80316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.680562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0010c5-c776-41b3-9d81-68856bb80316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.681256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0010c5-c776-41b3-9d81-68856bb80316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:56.682943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0010c5-c776-41b3-9d81-68856bb80316 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.693683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1202/2438] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:09:56.698006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:56.710073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b9de-cbc3-4c47-bcd8-a919fcbccd0a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:56.714736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:56.715462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.715739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.716009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] PATH_INFO: `/v3/projects/d1fcf58794fc47738ed845189904fcfa/users/36369de0ba02413bb685985412adbf6f/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.718044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1fcf58794fc47738ed845189904fcfa, user_id=36369de0ba02413bb685985412adbf6f, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:56.720976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.732309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1237/2439] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:09:56 2022] PUT /identity/v3/projects/d1fcf58794fc47738ed845189904fcfa/users/36369de0ba02413bb685985412adbf6f/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:09:56.744765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15456f4-cfc1-4c76-9d64-f04a4e4586b0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.745070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15456f4-cfc1-4c76-9d64-f04a4e4586b0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.745354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15456f4-cfc1-4c76-9d64-f04a4e4586b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.771620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:09:56.777474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a15456f4-cfc1-4c76-9d64-f04a4e4586b0 None None] MFA Rules not processed for user `36369de0ba02413bb685985412adbf6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:09:56.778720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a15456f4-cfc1-4c76-9d64-f04a4e4586b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:56.847126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1203/2440] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:09:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:09:56.860482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:56.863403 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:56.923665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869c19-94d1-4e7b-800b-a853c02bc362 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:56.928495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:56.928495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:56.928495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:56.928936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:56.938135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:56.939868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:56.949839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1238/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:57.180533 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15456f4-cfc1-4c76-9d64-f04a4e4586b0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:57.190683 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15456f4-cfc1-4c76-9d64-f04a4e4586b0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:57.194359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:57.194976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:57.195254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:57.195522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:57.255764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:57.257284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:57.266429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1204/2442] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:09:59.364124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:09:59.365940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:59.510733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61157185-362b-4f85-9c29-ecd4c0effa64 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:09:59.524469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f509c4-f97c-444b-b5d0-056916a997fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:09:59.525384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f509c4-f97c-444b-b5d0-056916a997fc service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:09:59.525784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f509c4-f97c-444b-b5d0-056916a997fc service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:09:59.526157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f509c4-f97c-444b-b5d0-056916a997fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:09:59.528083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a6f509c4-f97c-444b-b5d0-056916a997fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:09:59.540977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f509c4-f97c-444b-b5d0-056916a997fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:09:59.542591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f509c4-f97c-444b-b5d0-056916a997fc service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:09:59.555547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1239/2443] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:09:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:10:02.816438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:02.835099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcefc8b-6dc7-40c4-b002-1010a4417d9f service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:02.842141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:02.845776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:02.845776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:02.845776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:02.846367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:02.860928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:02.865290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:02.888898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1205/2444] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:10:08.996160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:09.001481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:09.001481 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:09.020365 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:09.021601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:09.055521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:09.078309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1240/2445] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:10:09.137718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:09.139524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:09.265946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cafc45-8c9a-4354-8f00-dda563760c87 service nova] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:09.271880 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:09.272708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:09.273064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:09.273451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:09.283673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:09.286495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:09.303417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1206/2446] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:10:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:10:12.663550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:12.663550 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:12.691399 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a210a5-ae7e-4043-827f-4e85f9cb0c74 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:12.696729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:12.697754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:12.698138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:12.698566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] PATH_INFO: `/v3/users/4be79b35cb3141689838d2b637ddf82c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:12.700049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be79b35cb3141689838d2b637ddf82c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:12.702554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:12.866467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:12.885395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8867b-2950-4509-a64f-8d5f9a81c83f service swift] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:12.903412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:12.904069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:12.904352 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:12.904615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] PATH_INFO: `/v3/users/d952d186196846f78d586ebd0353b747` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:12.910417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d952d186196846f78d586ebd0353b747)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:12.913327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:12.918131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:12.999875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be79b35cb3141689838d2b637ddf82c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.001322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be79b35cb3141689838d2b637ddf82c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.001869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be79b35cb3141689838d2b637ddf82c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.013738 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be79b35cb3141689838d2b637ddf82c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.086265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6cc05080-7adc-443a-a1c4-893b5918f8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be79b35cb3141689838d2b637ddf82c'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.119833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1241/2447] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:10:12 2022] DELETE /identity/v3/users/4be79b35cb3141689838d2b637ddf82c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:13.125959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:13.126638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:13.126991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:13.143233 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd952d186196846f78d586ebd0353b747'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.144259 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd952d186196846f78d586ebd0353b747'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.145087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd952d186196846f78d586ebd0353b747'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.150020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:13.152591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:13.235857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1242/2448] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:10:13.239711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd952d186196846f78d586ebd0353b747'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.247462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:13.250182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:13.257030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a2038a90-e7cc-4a74-967d-48c06e4e605d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd952d186196846f78d586ebd0353b747'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:13.261527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:13.305953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1207/2449] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:10:12 2022] DELETE /identity/v3/users/d952d186196846f78d586ebd0353b747 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:10:13.309168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:13.309657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:13.310102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:13.336423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490d941-28ae-4cc3-a7bb-e1551fc8bb00 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:13.339760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:13.341504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:13.438342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1208/2450] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:13.452567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:13.453413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:13.453702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:13.453980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:13.465349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:13.466918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:13.479008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1243/2451] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:10:13.497002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:13.498557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:13.541160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc1efb-6e53-4a9b-8a48-3cece0d7e530 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:13.550480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:13.550480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:13.550480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:13.550480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:13.566787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:13.568490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:13.586892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1209/2452] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:10:14.265694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:14.267295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:14.296742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1429c33-9202-4055-bc17-69379c3e6fbf service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:14.297149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:14.299127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:14.313825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:14.314840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:14.317852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:14.318197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] PATH_INFO: `/v3/projects/496d4105ca2e443bb4fcbd76e8ce0537` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:14.319715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=496d4105ca2e443bb4fcbd76e8ce0537)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:14.322346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cc7bf-96e0-4636-8e3f-6ef2e8b9ddc6 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:14.324379 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:14.328250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:14.329571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:14.330025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:14.330480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] PATH_INFO: `/v3/projects/291865d4372641af8d60c6cd6e8fc0dc` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:14.332593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=291865d4372641af8d60c6cd6e8fc0dc)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:14.335162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:14.454347 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496d4105ca2e443bb4fcbd76e8ce0537', 'request_id': 'req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.462193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496d4105ca2e443bb4fcbd76e8ce0537', 'request_id': 'req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.471276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291865d4372641af8d60c6cd6e8fc0dc', 'request_id': 'req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.474051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496d4105ca2e443bb4fcbd76e8ce0537', 'request_id': 'req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.479419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291865d4372641af8d60c6cd6e8fc0dc', 'request_id': 'req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.487619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291865d4372641af8d60c6cd6e8fc0dc', 'request_id': 'req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.491075 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:14.558933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] The token cache is being invalidate because project 496d4105ca2e443bb4fcbd76e8ce0537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:10:14.559277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.559861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.561393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1244/2453] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:10:14 2022] DELETE /identity/v3/projects/496d4105ca2e443bb4fcbd76e8ce0537 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:14.624615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] The token cache is being invalidate because project 291865d4372641af8d60c6cd6e8fc0dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:10:14.625430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.626516 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-21829d9c-cb8e-40a8-846b-20d90b1c2dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:14.632484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1210/2454] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:10:14 2022] DELETE /identity/v3/projects/291865d4372641af8d60c6cd6e8fc0dc => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:10:14.634352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:14.636146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:14.716192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:14.740510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fcf9a-acf0-4dd0-9005-2c6cea27bb5f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:14.862678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:14.863741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:14.864088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:14.864443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] PATH_INFO: `/v3/users/36369de0ba02413bb685985412adbf6f` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:14.868798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36369de0ba02413bb685985412adbf6f)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:14.872868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:14.991245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:14.991533 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:14.991814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:15.033225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:15.036578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:15.100497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36369de0ba02413bb685985412adbf6f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:15.101942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36369de0ba02413bb685985412adbf6f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:15.102700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36369de0ba02413bb685985412adbf6f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:15.115392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36369de0ba02413bb685985412adbf6f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:15.135551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1211/2455] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 22 11:10:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:10:15.144141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:15.146518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:15.193571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36369de0ba02413bb685985412adbf6f'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:15.217893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1245/2456] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:10:14 2022] DELETE /identity/v3/users/36369de0ba02413bb685985412adbf6f => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:15.238467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1786c00-1c67-4902-bb90-a78745763384 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:15.320096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1a1e2-ffb9-4497-b532-5b637d6efd77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:15.321163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1a1e2-ffb9-4497-b532-5b637d6efd77 service swift] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:15.322391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1a1e2-ffb9-4497-b532-5b637d6efd77 service swift] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:15.322391 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1a1e2-ffb9-4497-b532-5b637d6efd77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:15.371990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:15.391012 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1a1e2-ffb9-4497-b532-5b637d6efd77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:15.392893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1a1e2-ffb9-4497-b532-5b637d6efd77 service swift] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:15.404058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1212/2457] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 11:10:15.693471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:15.695170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:15.783115 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d641e12-a580-4cfb-a33d-11e1dcb27dd7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:15.788363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:15.789124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:15.789478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:15.789815 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] PATH_INFO: `/v3/projects/d1fcf58794fc47738ed845189904fcfa` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:15.791176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1fcf58794fc47738ed845189904fcfa)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:15.793175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:15.812018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:15.935305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1fcf58794fc47738ed845189904fcfa', 'request_id': 'req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:15.943215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1fcf58794fc47738ed845189904fcfa', 'request_id': 'req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:15.950653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1fcf58794fc47738ed845189904fcfa', 'request_id': 'req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:16.028492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] The token cache is being invalidate because project d1fcf58794fc47738ed845189904fcfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:10:16.028833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:16.029441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5dc1aa8-d683-4532-a808-6ea1ea8c1a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:16.031033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1246/2458] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:10:15 2022] DELETE /identity/v3/projects/d1fcf58794fc47738ed845189904fcfa => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:16.038746 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a0890-4794-488e-a70d-68e02bfde340 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.039378 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a0890-4794-488e-a70d-68e02bfde340 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.039728 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a0890-4794-488e-a70d-68e02bfde340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.063862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a32a0890-4794-488e-a70d-68e02bfde340 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:16.065373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a32a0890-4794-488e-a70d-68e02bfde340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:16.182304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1213/2459] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:16.188253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4355-1101-46e2-92c4-ecc57b3169a1 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.188253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4355-1101-46e2-92c4-ecc57b3169a1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.188253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd4355-1101-46e2-92c4-ecc57b3169a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.209395 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-09dd4355-1101-46e2-92c4-ecc57b3169a1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:16.210730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-09dd4355-1101-46e2-92c4-ecc57b3169a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:16.225428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1247/2460] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:16.226690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a0890-4794-488e-a70d-68e02bfde340 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:16.238737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a0890-4794-488e-a70d-68e02bfde340 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:16.269536 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:16.270432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.270695 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.271200 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.271897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:10:16.272284 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:16.273883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:16.290555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1214/2461] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:10:16.294735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4355-1101-46e2-92c4-ecc57b3169a1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:16.304968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd4355-1101-46e2-92c4-ecc57b3169a1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:16.309864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:16.310605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.310889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.311164 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.311953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:16.313910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:16.511237 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1248/2462] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:10:16 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:16.515347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:16.686299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33649e9-5f81-4901-bd21-f7eaeab542ef admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:16.686299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:16.686299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.686299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.686743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.686743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:16.686743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:16.851616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1215/2463] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:10:16 2022] POST /identity/v3/users => generated 288 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:16.859285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:16.870663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c7fc1-046e-4ccb-89be-0f3755959f8d admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:16.875292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:16.880901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.880901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.880901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.880901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:16.880901 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:16.889977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1249/2464] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:10:16.894111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:16.904771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84de26-93f2-4596-b717-d38462fce0cd admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:16.909802 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110d12b-9f41-4fcd-9e24-e802a9cf58f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:16.910448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d12b-9f41-4fcd-9e24-e802a9cf58f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.910888 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d12b-9f41-4fcd-9e24-e802a9cf58f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.910995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110d12b-9f41-4fcd-9e24-e802a9cf58f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.911634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d12b-9f41-4fcd-9e24-e802a9cf58f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:16.913228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110d12b-9f41-4fcd-9e24-e802a9cf58f8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:16.927246 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1216/2465] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:10:16.931566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:16.941597 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a64d46-6d21-4286-904d-4f129e6a7e5e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:16.946744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:16.947420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:16.947703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:16.947979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] PATH_INFO: `/v3/projects/2a13fcd587ba4843982e1d5854c02ed7/users/07877e56f34b4a24a0c5e23d980fac88/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:16.963381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a13fcd587ba4843982e1d5854c02ed7, user_id=07877e56f34b4a24a0c5e23d980fac88, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:16.966007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:17.062336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1250/2466] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:10:16 2022] PUT /identity/v3/projects/2a13fcd587ba4843982e1d5854c02ed7/users/07877e56f34b4a24a0c5e23d980fac88/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:17.073175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0680-0b00-421f-b53a-23e48a4790c7 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:17.073575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0680-0b00-421f-b53a-23e48a4790c7 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:17.073842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0680-0b00-421f-b53a-23e48a4790c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:17.104309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3fab0680-0b00-421f-b53a-23e48a4790c7 None None] MFA Rules not processed for user `07877e56f34b4a24a0c5e23d980fac88`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:17.105430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3fab0680-0b00-421f-b53a-23e48a4790c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:17.152777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:17.171722 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1217/2467] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:17.176730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:17.256977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd691f17-3c47-4d1a-9986-13946c4a4152 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:17.262506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:17.263336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:17.263719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:17.264045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:17.264787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:17.266851 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:17.353487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1251/2468] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:10:17 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:17.358341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab0680-0b00-421f-b53a-23e48a4790c7 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:17.374872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab0680-0b00-421f-b53a-23e48a4790c7 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:17.474264 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:17.474994 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:17.480959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:17.480959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:17.480959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:17.480959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:17.589160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1218/2469] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:10:17 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:17.594760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:17.607043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edef9d1-66b2-4463-a635-5349735e06e7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:17.617889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:17.618470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:17.618741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:17.619051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:17.619658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:17.621272 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:17.652290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1252/2470] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:10:17.656041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:17.673657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc590d1-b7fc-497f-86e6-88957738cab0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:17.681763 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:17.682987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:17.683629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:17.684142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] PATH_INFO: `/v3/projects/76d181ce87b54538951a9a2f8bfc699b/users/eb31188f93364aa1b5470d880d611241/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:17.709910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d181ce87b54538951a9a2f8bfc699b, user_id=eb31188f93364aa1b5470d880d611241, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:17.712823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:17.803968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1219/2471] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:10:17 2022] PUT /identity/v3/projects/76d181ce87b54538951a9a2f8bfc699b/users/eb31188f93364aa1b5470d880d611241/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:17.807975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:17.821631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b431ff-c8d3-4944-887e-4445ce16ee28 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:17.850104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:17.930601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29b908-ea8d-4637-bb45-362aee0d5d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:17.932834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29b908-ea8d-4637-bb45-362aee0d5d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:17.934632 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29b908-ea8d-4637-bb45-362aee0d5d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:17.934948 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29b908-ea8d-4637-bb45-362aee0d5d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:17.935620 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29b908-ea8d-4637-bb45-362aee0d5d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:17.938289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29b908-ea8d-4637-bb45-362aee0d5d30 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:17.967433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1253/2472] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:10:17.972213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:17.991290 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f53df-9073-4909-85cc-af94c13370b7 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:17.996183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:17.996862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:17.997138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:17.997406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] PATH_INFO: `/v3/projects/76d181ce87b54538951a9a2f8bfc699b/users/eb31188f93364aa1b5470d880d611241/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:18.000311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d181ce87b54538951a9a2f8bfc699b, user_id=eb31188f93364aa1b5470d880d611241, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:18.003006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:18.032926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1220/2473] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:10:17 2022] PUT /identity/v3/projects/76d181ce87b54538951a9a2f8bfc699b/users/eb31188f93364aa1b5470d880d611241/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:18.038769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:18.039059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:18.039340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:18.076594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] MFA Rules not processed for user `eb31188f93364aa1b5470d880d611241`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:18.077870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:18.194406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1254/2474] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:18.201911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:18.204670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:18.291574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965638c0-604a-46fe-92c8-ef9036d7c6b7 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:18.301943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:18.302993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:18.303449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:18.303879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:18.367824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:18.381325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:18.382823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:18.406265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1221/2475] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:10:25.550150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:25.552292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:25.571526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95875bdd-1318-4cf7-a25c-83ac2242207e None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:25.576266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:25.576991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:25.577192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:25.577493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:25.590291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:25.591789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:25.618405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1255/2476] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:10:26.220323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:26.222373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:26.303417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a31280-695e-41c0-8b9c-39579a7a1a54 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:26.308711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:26.309452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:26.309787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:26.310293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] PATH_INFO: `/v3/users/93ec72fc30ea45d5952bf9f29311e4ac` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:26.311611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ec72fc30ea45d5952bf9f29311e4ac)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:26.313602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:26.573561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ec72fc30ea45d5952bf9f29311e4ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:26.574126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ec72fc30ea45d5952bf9f29311e4ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:26.574767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ec72fc30ea45d5952bf9f29311e4ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:26.645535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ec72fc30ea45d5952bf9f29311e4ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:26.652976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ec72fc30ea45d5952bf9f29311e4ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:26.670388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1222/2477] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:10:26 2022] DELETE /identity/v3/users/93ec72fc30ea45d5952bf9f29311e4ac => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:26.892839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:26.907560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d14de0-35a0-491b-add6-95798219e843 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:26.960702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:26.961327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:26.961608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:26.961880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] PATH_INFO: `/v3/projects/0533d69c876f4cbdbf6b7138abf37893` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:26.963024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0533d69c876f4cbdbf6b7138abf37893)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:26.964911 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:26.989047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:27.090153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0533d69c876f4cbdbf6b7138abf37893', 'request_id': 'req-90ad8250-e80a-4889-9f7a-b199a37b9c80'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.096174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0533d69c876f4cbdbf6b7138abf37893', 'request_id': 'req-90ad8250-e80a-4889-9f7a-b199a37b9c80'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.102971 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0533d69c876f4cbdbf6b7138abf37893', 'request_id': 'req-90ad8250-e80a-4889-9f7a-b199a37b9c80'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.164701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] The token cache is being invalidate because project 0533d69c876f4cbdbf6b7138abf37893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:10:27.164927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.165548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.167089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1256/2478] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:10:26 2022] DELETE /identity/v3/projects/0533d69c876f4cbdbf6b7138abf37893 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:27.171616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:27.173023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:27.186628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e6077-7f70-45de-8fd5-cf2487df7668 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:27.244093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:27.244724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:27.244999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:27.245272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] PATH_INFO: `/v3/users/9894bf1d999e4739ab1aec2ffd4387bd` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:27.246433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9894bf1d999e4739ab1aec2ffd4387bd)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:27.248363 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:27.253286 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:27.401444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9894bf1d999e4739ab1aec2ffd4387bd'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.402099 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9894bf1d999e4739ab1aec2ffd4387bd'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.402734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9894bf1d999e4739ab1aec2ffd4387bd'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.468367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9894bf1d999e4739ab1aec2ffd4387bd'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.480469 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a252f856-f32d-4c1e-a22f-e0e73f1ce2d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9894bf1d999e4739ab1aec2ffd4387bd'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.505596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1223/2479] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:10:27 2022] DELETE /identity/v3/users/9894bf1d999e4739ab1aec2ffd4387bd => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:27.765520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:27.767090 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:27.781153 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad8250-e80a-4889-9f7a-b199a37b9c80 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:27.836833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:27.837465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:27.837748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:27.838023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] PATH_INFO: `/v3/projects/78ef109fc3a24910b73a08434559a8f6` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:27.839199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ef109fc3a24910b73a08434559a8f6)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:27.841094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:27.942113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ef109fc3a24910b73a08434559a8f6', 'request_id': 'req-8ef69061-6946-4e03-959a-b976f98f7994'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.948473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ef109fc3a24910b73a08434559a8f6', 'request_id': 'req-8ef69061-6946-4e03-959a-b976f98f7994'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:27.954601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ef109fc3a24910b73a08434559a8f6', 'request_id': 'req-8ef69061-6946-4e03-959a-b976f98f7994'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:28.055961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] The token cache is being invalidate because project 78ef109fc3a24910b73a08434559a8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:10:28.056171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:28.056818 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:28.058358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1257/2480] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:10:27 2022] DELETE /identity/v3/projects/78ef109fc3a24910b73a08434559a8f6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:10:37.015449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:37.015940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:37.016324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:37.038543 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:37.039760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:37.093960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1224/2481] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:10:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:37.106674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:37.108089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:37.153734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:37.177894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef69061-6946-4e03-959a-b976f98f7994 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:37.182955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3792c-bd05-4bf5-80a7-3a7cc72b11d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:37.183562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3792c-bd05-4bf5-80a7-3a7cc72b11d2 service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:37.183843 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3792c-bd05-4bf5-80a7-3a7cc72b11d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:37.184119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3792c-bd05-4bf5-80a7-3a7cc72b11d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:37.192140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3792c-bd05-4bf5-80a7-3a7cc72b11d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:37.193778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3792c-bd05-4bf5-80a7-3a7cc72b11d2 service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:37.201715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1258/2482] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:10:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:10:47.087343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:47.336329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=13 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:53.978755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:53.980217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:54.045793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58e04b-83f7-4bf1-bbcb-0c191019577b None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:54.050333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:54.050965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:54.051393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:54.051674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] PATH_INFO: `/v3/users/07877e56f34b4a24a0c5e23d980fac88` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:54.052846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07877e56f34b4a24a0c5e23d980fac88)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:54.054897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:54.224144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07877e56f34b4a24a0c5e23d980fac88'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:54.224846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07877e56f34b4a24a0c5e23d980fac88'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:54.226869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07877e56f34b4a24a0c5e23d980fac88'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:54.297691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07877e56f34b4a24a0c5e23d980fac88'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:54.308193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4d97a3b4-12d4-43d5-8e21-5f63762f875c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07877e56f34b4a24a0c5e23d980fac88'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:54.320979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:54.321494 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:54.321600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:54.328199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1225/2483] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:10:53 2022] DELETE /identity/v3/users/07877e56f34b4a24a0c5e23d980fac88 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:54.332606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:54.332891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:54.333164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:54.340898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:54.342121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:54.352122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:54.353298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:54.401813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1259/2484] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 22 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:10:54.415092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:54.416548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:54.416778 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1226/2485] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:54.426459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:54.428310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:54.450578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:54.489077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540b6e8-0de3-4e07-80d8-5675cb1ccfbe None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:54.489872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da9a56-8632-4973-9c3b-5529148bec1a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:54.493840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:54.496389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:54.496389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:54.496389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:54.496803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:54.496803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:54.496803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:54.496803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:54.504851 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:54.505010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:54.507068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:54.507685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:54.515902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1260/2486] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:10:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 22 11:10:54.524253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1227/2487] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:10:54.944584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:54.946035 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:54.962350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f649667-96d0-45b1-832b-c66e4c040a5d service swift] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:54.969386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:54.970083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:54.970394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:54.970667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] PATH_INFO: `/v3/projects/2a13fcd587ba4843982e1d5854c02ed7` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:54.971819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a13fcd587ba4843982e1d5854c02ed7)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:54.973706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:55.113074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a13fcd587ba4843982e1d5854c02ed7', 'request_id': 'req-7ef894c1-df15-477c-a30d-f48f221b27f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.115899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a13fcd587ba4843982e1d5854c02ed7', 'request_id': 'req-7ef894c1-df15-477c-a30d-f48f221b27f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.122342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a13fcd587ba4843982e1d5854c02ed7', 'request_id': 'req-7ef894c1-df15-477c-a30d-f48f221b27f3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.189240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] The token cache is being invalidate because project 2a13fcd587ba4843982e1d5854c02ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:10:55.189655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.190321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.192032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1261/2488] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:10:54 2022] DELETE /identity/v3/projects/2a13fcd587ba4843982e1d5854c02ed7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:55.196448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:55.206360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef163f68-db77-4d9f-9d14-6557c466dbfc service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:55.263521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:55.264123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:55.264410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:55.264685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] PATH_INFO: `/v3/users/eb31188f93364aa1b5470d880d611241` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:55.265836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb31188f93364aa1b5470d880d611241)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:55.267827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:55.454034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb31188f93364aa1b5470d880d611241'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.454820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb31188f93364aa1b5470d880d611241'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.455430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb31188f93364aa1b5470d880d611241'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.514168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb31188f93364aa1b5470d880d611241'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.521830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19c9f506-d8e1-44e2-b390-d79e3e3f84f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb31188f93364aa1b5470d880d611241'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:55.539662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1228/2489] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:10:55 2022] DELETE /identity/v3/users/eb31188f93364aa1b5470d880d611241 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:55.921775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:55.923945 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:55.940302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef894c1-df15-477c-a30d-f48f221b27f3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:55.954444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:55.998064 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:55.998723 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:55.998999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:55.999297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] PATH_INFO: `/v3/projects/76d181ce87b54538951a9a2f8bfc699b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.000447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d181ce87b54538951a9a2f8bfc699b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:56.002397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:56.130979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d181ce87b54538951a9a2f8bfc699b', 'request_id': 'req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:56.136854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d181ce87b54538951a9a2f8bfc699b', 'request_id': 'req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:56.142902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d181ce87b54538951a9a2f8bfc699b', 'request_id': 'req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:56.228306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] The token cache is being invalidate because project 76d181ce87b54538951a9a2f8bfc699b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:10:56.228592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:56.229211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66ddf3af-d614-46aa-9bd1-0e4dfddb961b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:10:56.230751 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1262/2490] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:10:55 2022] DELETE /identity/v3/projects/76d181ce87b54538951a9a2f8bfc699b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:56.235675 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7b502-8d10-4852-b21f-3ddfe5cec986 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.236150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7b502-8d10-4852-b21f-3ddfe5cec986 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.236340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7b502-8d10-4852-b21f-3ddfe5cec986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.256701 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-cee7b502-8d10-4852-b21f-3ddfe5cec986 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:56.258587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-cee7b502-8d10-4852-b21f-3ddfe5cec986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:56.299490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:56.323572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1229/2491] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:56.331478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16579c2-5aed-4290-873a-81ae7f9b84e4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.331767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16579c2-5aed-4290-873a-81ae7f9b84e4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.332036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16579c2-5aed-4290-873a-81ae7f9b84e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.353227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c16579c2-5aed-4290-873a-81ae7f9b84e4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:56.354483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c16579c2-5aed-4290-873a-81ae7f9b84e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:56.370177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1263/2492] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:56.379918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7b502-8d10-4852-b21f-3ddfe5cec986 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:56.392897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7b502-8d10-4852-b21f-3ddfe5cec986 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:56.392897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:56.392897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.392897 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.393221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.393221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:10:56.393221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:56.393961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:56.406956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1230/2493] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:10:56.411591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16579c2-5aed-4290-873a-81ae7f9b84e4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:56.421759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16579c2-5aed-4290-873a-81ae7f9b84e4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:56.426466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:56.427072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.427350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.427628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.428378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:56.430873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:56.538165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1264/2494] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:10:56 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:56.542439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:56.552333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e370bc0-7f48-44b4-aa73-82af654ddc81 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:56.607954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:56.608555 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.608844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.609116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.609727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:56.611705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:56.714068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1231/2495] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:10:56 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:56.718502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:56.729598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad9a53-fc18-4cfa-8a70-3564b0c72c07 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:56.734215 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:56.734871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.735219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.735498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.736250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:56.737861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:56.749466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1265/2496] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:10:56.753372 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:56.764159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f489-8796-4503-b94d-5b130955c802 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:56.768813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bb2a5-82b5-4db2-b239-7162a78c608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:56.769408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bb2a5-82b5-4db2-b239-7162a78c608a admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.769684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bb2a5-82b5-4db2-b239-7162a78c608a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.769952 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bb2a5-82b5-4db2-b239-7162a78c608a admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.770615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bb2a5-82b5-4db2-b239-7162a78c608a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:56.772208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bb2a5-82b5-4db2-b239-7162a78c608a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:56.784196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1232/2497] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:10:56.788990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:56.799351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800eaf80-6795-43b7-9730-9a844d2e9316 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:56.804720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:56.805412 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.805722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.805996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] PATH_INFO: `/v3/projects/c26fc67271bb4ad8ba37eac42cd84943/users/44a5c19963134edfbd5c1c42a6e737ee/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.820569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26fc67271bb4ad8ba37eac42cd84943, user_id=44a5c19963134edfbd5c1c42a6e737ee, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:56.823249 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:56.925865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1266/2498] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:10:56 2022] PUT /identity/v3/projects/c26fc67271bb4ad8ba37eac42cd84943/users/44a5c19963134edfbd5c1c42a6e737ee/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:56.938014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a597719-be27-4383-bc16-ae480afeae20 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:56.938410 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a597719-be27-4383-bc16-ae480afeae20 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:56.938686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a597719-be27-4383-bc16-ae480afeae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:56.974706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-6a597719-be27-4383-bc16-ae480afeae20 None None] MFA Rules not processed for user `44a5c19963134edfbd5c1c42a6e737ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:56.976485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6a597719-be27-4383-bc16-ae480afeae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:57.052850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1233/2499] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:57.054927 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:57.103059 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:57.120572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60ba82-d7dc-4466-9191-4f723891e57a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:57.125705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:57.126349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.126633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.126906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.127541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:57.129490 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:57.247477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1267/2500] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:10:57 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:57.251243 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a597719-be27-4383-bc16-ae480afeae20 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:57.261858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a597719-be27-4383-bc16-ae480afeae20 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:57.295210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:57.317768 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:57.318405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.318684 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.318955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.319611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:57.321532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:57.410371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1234/2501] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:10:57 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:57.414533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:57.424860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cda92-f8cd-404c-a099-946bd196d37a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:57.430687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:57.431308 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.438556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.438556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.438556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:57.438556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:57.445228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1268/2502] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:10:57.449253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:57.460013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1eccd-5071-4e2f-82ca-ae3cec69a0cd admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:57.464928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329efcb-2394-4466-9050-03392d17bb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:57.465526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329efcb-2394-4466-9050-03392d17bb0f admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.465805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329efcb-2394-4466-9050-03392d17bb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.466078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329efcb-2394-4466-9050-03392d17bb0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.466796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329efcb-2394-4466-9050-03392d17bb0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:57.468392 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329efcb-2394-4466-9050-03392d17bb0f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:57.478669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1235/2503] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:10:57.482682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:57.493860 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516fa6f-f30a-44f4-9482-1d01dbdec841 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:57.498684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:57.499276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.499555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.499861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] PATH_INFO: `/v3/projects/258165b1b018481c83f8eccd7029720c/users/ea0928a10e814887bd39873090d60daf/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.514435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=258165b1b018481c83f8eccd7029720c, user_id=ea0928a10e814887bd39873090d60daf, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:57.517206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:57.623616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1269/2504] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:10:57 2022] PUT /identity/v3/projects/258165b1b018481c83f8eccd7029720c/users/ea0928a10e814887bd39873090d60daf/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:10:57.634411 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6235-fd80-4183-a1e7-8406e2b8cd42 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.634698 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6235-fd80-4183-a1e7-8406e2b8cd42 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.634971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a6235-fd80-4183-a1e7-8406e2b8cd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.662340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4e3a6235-fd80-4183-a1e7-8406e2b8cd42 None None] MFA Rules not processed for user `ea0928a10e814887bd39873090d60daf`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:57.663548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4e3a6235-fd80-4183-a1e7-8406e2b8cd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:57.726451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1236/2505] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:57.731014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:57.793684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca982df0-7729-4320-abd4-386c100b68f2 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:57.798952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:57.799556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.799841 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.800117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.800741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:57.802674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:57.891151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1270/2506] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:10:57 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:57.892419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a6235-fd80-4183-a1e7-8406e2b8cd42 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:57.903890 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a6235-fd80-4183-a1e7-8406e2b8cd42 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:57.960918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:57.961529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:57.961803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:57.962072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:57.962710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:57.964708 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:58.093154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1237/2507] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:10:57 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:10:58.096484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:58.107498 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae3e16-77da-44a1-a314-6e64b7fd7014 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:58.112080 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:58.112886 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:58.113170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:58.113443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:58.114095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:58.116012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:58.126928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1271/2508] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:10:58.131248 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:58.142160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e5bbf-0ba5-4ef0-87c4-cc771645c9e9 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:58.147030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:58.148212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:58.148486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:58.148757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] PATH_INFO: `/v3/projects/5dc44879cc754b519c63870482122929/users/dee2e6f07bb94878b1da8dc3cd6b8aba/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:58.164529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc44879cc754b519c63870482122929, user_id=dee2e6f07bb94878b1da8dc3cd6b8aba, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:58.167167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:58.223195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1238/2509] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:10:58 2022] PUT /identity/v3/projects/5dc44879cc754b519c63870482122929/users/dee2e6f07bb94878b1da8dc3cd6b8aba/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:58.228039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:58.238123 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a89fb7-8468-44fc-a24f-5a5bdc8c9ae2 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:58.296659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ccdac-eb32-4c80-9357-1160410f097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:58.297273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ccdac-eb32-4c80-9357-1160410f097d admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:58.297559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ccdac-eb32-4c80-9357-1160410f097d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:58.297830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ccdac-eb32-4c80-9357-1160410f097d admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:58.298501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ccdac-eb32-4c80-9357-1160410f097d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:58.300111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ccdac-eb32-4c80-9357-1160410f097d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:58.310729 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1272/2510] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:10:58.315025 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:58.325506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3808-f40c-4bef-9a4a-e11774c870cc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:58.330399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:58.331043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:58.331322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:58.331593 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] PATH_INFO: `/v3/projects/5dc44879cc754b519c63870482122929/users/dee2e6f07bb94878b1da8dc3cd6b8aba/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:58.333465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc44879cc754b519c63870482122929, user_id=dee2e6f07bb94878b1da8dc3cd6b8aba, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:58.337370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:58.342935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:58.351714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1239/2511] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:10:58 2022] PUT /identity/v3/projects/5dc44879cc754b519c63870482122929/users/dee2e6f07bb94878b1da8dc3cd6b8aba/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:10:58.360367 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:58.360500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:58.360766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:58.369651 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:10:58.392033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] MFA Rules not processed for user `dee2e6f07bb94878b1da8dc3cd6b8aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:10:58.393248 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:58.461398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1273/2512] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:10:58.475249 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:58.477497 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:58.533908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805672a-f378-405b-9090-7765d3eeba61 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:58.537609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:58.538836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:58.539240 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:58.539645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:58.549940 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:58.551537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:58.562015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1240/2513] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:10:59.635881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:10:59.635881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:10:59.639910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa1305-2c6d-4d60-8902-2fd6785d34c4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:10:59.643515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:10:59.644115 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:10:59.644394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:10:59.644666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:10:59.700584 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:10:59.702082 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:10:59.711551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1274/2514] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:10:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:04.650738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:04.659710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af8a4-ad10-4ccd-bb0a-961359f19366 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:04.664120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:04.664725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:04.665022 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:04.665295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:04.723019 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:04.724457 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:04.733540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1241/2515] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:05.869574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:05.871240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:05.942119 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9e4bd-69ef-4620-ab20-3cb724779105 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:05.945833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0658c25-3eed-4575-a79e-b8277c489f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:05.946480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0658c25-3eed-4575-a79e-b8277c489f95 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:05.946760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0658c25-3eed-4575-a79e-b8277c489f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:05.947032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0658c25-3eed-4575-a79e-b8277c489f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:05.948795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a0658c25-3eed-4575-a79e-b8277c489f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:05.997633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:06.015086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0658c25-3eed-4575-a79e-b8277c489f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:06.016583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0658c25-3eed-4575-a79e-b8277c489f95 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:06.026433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1275/2516] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:08.533476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:08.604107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f6b5-8257-473c-b21b-b59c807cf2ce service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:08.611035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:08.611949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:08.612303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:08.612624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] PATH_INFO: `/v3/users/44a5c19963134edfbd5c1c42a6e737ee` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:08.614597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a5c19963134edfbd5c1c42a6e737ee)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:08.617492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:08.837731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:08.846960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a5c19963134edfbd5c1c42a6e737ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:08.847664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a5c19963134edfbd5c1c42a6e737ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:08.848305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a5c19963134edfbd5c1c42a6e737ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:08.902654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a5c19963134edfbd5c1c42a6e737ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:08.910715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a5c19963134edfbd5c1c42a6e737ee'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:08.928300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1242/2517] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:08 2022] DELETE /identity/v3/users/44a5c19963134edfbd5c1c42a6e737ee => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:08.933217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:08.937966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:08.937966 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:08.955889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:08.957210 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:09.033326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1276/2518] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:09.063996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:09.065571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:09.129743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cd9fc-f429-4bbe-b27a-817988ccb121 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:09.134186 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:09.134849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:09.135128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:09.135395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:09.146324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:09.147864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:09.157442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1243/2519] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:09.718039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:09.719707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:09.734352 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2feb-919d-4e7a-a972-2edf1190e042 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:09.739127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:09.739765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:09.740339 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:09.740616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] PATH_INFO: `/v3/projects/c26fc67271bb4ad8ba37eac42cd84943` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:09.742802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26fc67271bb4ad8ba37eac42cd84943)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:09.744480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:09.839810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26fc67271bb4ad8ba37eac42cd84943', 'request_id': 'req-af57c897-64c4-4acf-bb11-5c28b4f1d38a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:09.845348 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26fc67271bb4ad8ba37eac42cd84943', 'request_id': 'req-af57c897-64c4-4acf-bb11-5c28b4f1d38a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:09.853184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26fc67271bb4ad8ba37eac42cd84943', 'request_id': 'req-af57c897-64c4-4acf-bb11-5c28b4f1d38a'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:09.902795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] The token cache is being invalidate because project c26fc67271bb4ad8ba37eac42cd84943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:09.903039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:09.903641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:09.905133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1277/2520] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:09 2022] DELETE /identity/v3/projects/c26fc67271bb4ad8ba37eac42cd84943 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:09.909134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:09.920039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63c99d-8c43-43fa-aca8-3f8d9de77901 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:09.977133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:09.977975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:09.978283 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:09.978567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] PATH_INFO: `/v3/users/ea0928a10e814887bd39873090d60daf` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:09.979737 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0928a10e814887bd39873090d60daf)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:09.981720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:10.243513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0928a10e814887bd39873090d60daf'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:10.244156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0928a10e814887bd39873090d60daf'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:10.244755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0928a10e814887bd39873090d60daf'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:10.354607 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0928a10e814887bd39873090d60daf'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:10.362612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0928a10e814887bd39873090d60daf'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:10.364124 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:10.383681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1244/2521] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:09 2022] DELETE /identity/v3/users/ea0928a10e814887bd39873090d60daf => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:10.829547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:10.839993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:10.848908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57c897-64c4-4acf-bb11-5c28b4f1d38a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:10.917171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:10.917816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:10.918098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:10.918400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] PATH_INFO: `/v3/projects/258165b1b018481c83f8eccd7029720c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:10.919516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=258165b1b018481c83f8eccd7029720c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:10.921499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:10.932141 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:11.039570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '258165b1b018481c83f8eccd7029720c', 'request_id': 'req-6bb6249d-d38a-4efd-a839-3361c60b26c8'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.045257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '258165b1b018481c83f8eccd7029720c', 'request_id': 'req-6bb6249d-d38a-4efd-a839-3361c60b26c8'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.051548 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '258165b1b018481c83f8eccd7029720c', 'request_id': 'req-6bb6249d-d38a-4efd-a839-3361c60b26c8'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.165480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] The token cache is being invalidate because project 258165b1b018481c83f8eccd7029720c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:11.165771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.166422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.167924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1278/2522] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:10 2022] DELETE /identity/v3/projects/258165b1b018481c83f8eccd7029720c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:11.172297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:11.173724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:11.186795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa638bc8-70c8-43ba-9af0-b98e4f7c7699 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:11.243169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:11.243797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:11.244074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:11.244344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] PATH_INFO: `/v3/users/dee2e6f07bb94878b1da8dc3cd6b8aba` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:11.245530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=dee2e6f07bb94878b1da8dc3cd6b8aba)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:11.247455 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:11.384050 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dee2e6f07bb94878b1da8dc3cd6b8aba'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.384689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dee2e6f07bb94878b1da8dc3cd6b8aba'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.385340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dee2e6f07bb94878b1da8dc3cd6b8aba'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.446492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dee2e6f07bb94878b1da8dc3cd6b8aba'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.454127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f5a6618a-7d3c-4539-9fb8-534313ee14ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dee2e6f07bb94878b1da8dc3cd6b8aba'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:11.471062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1245/2523] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:11 2022] DELETE /identity/v3/users/dee2e6f07bb94878b1da8dc3cd6b8aba => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:11.882370 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:11.887844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:11.898547 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6249d-d38a-4efd-a839-3361c60b26c8 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:11.960371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:11.961018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:11.961297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:11.961569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] PATH_INFO: `/v3/projects/5dc44879cc754b519c63870482122929` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:11.962760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc44879cc754b519c63870482122929)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:11.964654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:12.075096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc44879cc754b519c63870482122929', 'request_id': 'req-170d32c7-35af-4718-9800-01c2380fb5e5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:12.080517 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc44879cc754b519c63870482122929', 'request_id': 'req-170d32c7-35af-4718-9800-01c2380fb5e5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:12.087217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc44879cc754b519c63870482122929', 'request_id': 'req-170d32c7-35af-4718-9800-01c2380fb5e5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:12.169791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] The token cache is being invalidate because project 5dc44879cc754b519c63870482122929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:12.170015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:12.170658 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-170d32c7-35af-4718-9800-01c2380fb5e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:12.172162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1279/2524] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:11 2022] DELETE /identity/v3/projects/5dc44879cc754b519c63870482122929 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:12.178143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c7219-0b04-4a63-9c7f-9c7fd12f5cfe None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.179249 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c7219-0b04-4a63-9c7f-9c7fd12f5cfe None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.179249 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c7219-0b04-4a63-9c7f-9c7fd12f5cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.199199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ff0c7219-0b04-4a63-9c7f-9c7fd12f5cfe None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:12.200406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ff0c7219-0b04-4a63-9c7f-9c7fd12f5cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:12.255471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:12.264504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1246/2525] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:12.272705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773c99-3aa8-44e3-bebd-61501cc8cc2c None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.273241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773c99-3aa8-44e3-bebd-61501cc8cc2c None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.273514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773c99-3aa8-44e3-bebd-61501cc8cc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.300013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-16773c99-3aa8-44e3-bebd-61501cc8cc2c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:12.301520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-16773c99-3aa8-44e3-bebd-61501cc8cc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:12.313759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1280/2526] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:12.315215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c7219-0b04-4a63-9c7f-9c7fd12f5cfe None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:12.325835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c7219-0b04-4a63-9c7f-9c7fd12f5cfe None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:12.330620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:12.331242 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.331573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.331844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.332521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:12.332821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:12.334451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:12.348959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1247/2527] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:11:12.351239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16773c99-3aa8-44e3-bebd-61501cc8cc2c None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:12.360754 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16773c99-3aa8-44e3-bebd-61501cc8cc2c None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:12.365978 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:12.367344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.367830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.368167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.368872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:12.370894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:12.463016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:12.474771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1281/2528] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:12 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:12.476903 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:12.488449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dc36b-b051-4459-8f98-c85ad82d2ea0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:12.555215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:12.555936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.556289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.556630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.557327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:12.559387 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:12.657376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1248/2529] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:12 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:12.661957 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:12.673678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0462a0-0485-44e0-a57b-dbbdfb55bc3c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:12.679900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:12.680460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.680799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.681127 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.681836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:12.683715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:12.694921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1282/2530] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:12.699071 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:12.710419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91106a-737d-48a0-996a-15c7253da350 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:12.715137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308837a-4e1b-438e-85a8-6576c384f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:12.715738 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308837a-4e1b-438e-85a8-6576c384f12e admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.716820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308837a-4e1b-438e-85a8-6576c384f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.716820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308837a-4e1b-438e-85a8-6576c384f12e admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.717016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308837a-4e1b-438e-85a8-6576c384f12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:12.718799 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308837a-4e1b-438e-85a8-6576c384f12e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:12.730979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1249/2531] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:11:12.735004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:12.745825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a57e3e-d976-472c-a0a3-4202dfd94294 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:12.750434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:12.751130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.751409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.751682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] PATH_INFO: `/v3/projects/90bf301b365a4f3e97f78a382607263f/users/cfb0e614917f4937963bbf0ead963089/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.767475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bf301b365a4f3e97f78a382607263f, user_id=cfb0e614917f4937963bbf0ead963089, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:12.770158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:12.857905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1283/2532] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:12 2022] PUT /identity/v3/projects/90bf301b365a4f3e97f78a382607263f/users/cfb0e614917f4937963bbf0ead963089/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:12.870295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eccb9-baa8-4579-bd0e-5a2c3026139e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:12.870586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eccb9-baa8-4579-bd0e-5a2c3026139e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:12.870860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eccb9-baa8-4579-bd0e-5a2c3026139e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:12.901373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-3b4eccb9-baa8-4579-bd0e-5a2c3026139e None None] MFA Rules not processed for user `cfb0e614917f4937963bbf0ead963089`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:12.902858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3b4eccb9-baa8-4579-bd0e-5a2c3026139e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:12.980476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1250/2533] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:12.985438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:13.063028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194419a-2c2e-4b02-bdff-f8e147addf4f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:13.068802 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:13.070220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.070551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.070992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.071813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:13.074422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:13.212482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1284/2534] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:12 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:13.217119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eccb9-baa8-4579-bd0e-5a2c3026139e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:13.230861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eccb9-baa8-4579-bd0e-5a2c3026139e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:13.317114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:13.321965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:13.322620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.322896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.323164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.323794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:13.325770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:13.453238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1251/2535] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:13 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:13.457570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:13.471340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fbf9d5-f346-4534-a064-e060516cb952 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:13.477668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:13.478692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.479133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.479581 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.480410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:13.482314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:13.499621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1285/2536] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:13.505232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:13.531406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df829e-cef6-437d-bab3-bfc4446c1b7a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:13.531976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:13.534284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:13.536615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:13.537636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.538578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.538920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] PATH_INFO: `/v3/projects/7be9df759ead4cab85eee385c24b9388/users/35af368e680444c180d937bcc760567e/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.558276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be9df759ead4cab85eee385c24b9388, user_id=35af368e680444c180d937bcc760567e, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:13.560755 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:13.606962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390326-3329-4ae9-82bb-32858d2ff5df admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:13.613270 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:13.613756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.614037 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.614349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.616682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:13.630479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:13.631953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:13.642952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1286/2537] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:13.667444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1252/2538] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:13 2022] PUT /identity/v3/projects/7be9df759ead4cab85eee385c24b9388/users/35af368e680444c180d937bcc760567e/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:13.668659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:13.680853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725581e1-c044-443d-9ff1-fa751dfe8585 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:13.729821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:13.756760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e636f53-683c-465d-aebc-ad9a6e6a8460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:13.757999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636f53-683c-465d-aebc-ad9a6e6a8460 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.758501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636f53-683c-465d-aebc-ad9a6e6a8460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.758942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e636f53-683c-465d-aebc-ad9a6e6a8460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.760106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e636f53-683c-465d-aebc-ad9a6e6a8460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:13.761872 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e636f53-683c-465d-aebc-ad9a6e6a8460 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:13.773310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1287/2539] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:13.778278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:13.791668 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5682-bffb-4e42-a1c3-1364888e6622 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:13.796904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:13.797729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.798123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.798525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] PATH_INFO: `/v3/projects/7be9df759ead4cab85eee385c24b9388/users/35af368e680444c180d937bcc760567e/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.800686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be9df759ead4cab85eee385c24b9388, user_id=35af368e680444c180d937bcc760567e, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:13.803486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:13.821574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1253/2540] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:13 2022] PUT /identity/v3/projects/7be9df759ead4cab85eee385c24b9388/users/35af368e680444c180d937bcc760567e/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:13.895933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e7548-de39-449e-aba4-23c72e4f628d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:13.896310 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e7548-de39-449e-aba4-23c72e4f628d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:13.896669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e7548-de39-449e-aba4-23c72e4f628d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:13.941791 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-cf6e7548-de39-449e-aba4-23c72e4f628d None None] MFA Rules not processed for user `35af368e680444c180d937bcc760567e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:13.943054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cf6e7548-de39-449e-aba4-23c72e4f628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:14.029015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1288/2541] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:14.042397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:14.044764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:14.145731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb851253-9752-4be9-b0b4-eb960a10bbcf admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:14.151686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce733e-51f1-4255-b51e-00ea4f0aa854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:14.152726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce733e-51f1-4255-b51e-00ea4f0aa854 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:14.153193 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce733e-51f1-4255-b51e-00ea4f0aa854 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:14.153622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce733e-51f1-4255-b51e-00ea4f0aa854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:14.234476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce733e-51f1-4255-b51e-00ea4f0aa854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:14.235935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce733e-51f1-4255-b51e-00ea4f0aa854 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:14.246785 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1254/2542] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:17.416096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e7548-de39-449e-aba4-23c72e4f628d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:17.488515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e7548-de39-449e-aba4-23c72e4f628d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:17.495539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:17.496289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:17.496567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:17.496837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] PATH_INFO: `/v3/users/cfb0e614917f4937963bbf0ead963089` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:17.497995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb0e614917f4937963bbf0ead963089)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:17.499959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:17.679518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb0e614917f4937963bbf0ead963089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:17.680361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb0e614917f4937963bbf0ead963089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:17.681032 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb0e614917f4937963bbf0ead963089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:17.688884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb0e614917f4937963bbf0ead963089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:17.766433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb0e614917f4937963bbf0ead963089'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:17.768361 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:17.804760 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1289/2543] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:17 2022] DELETE /identity/v3/users/cfb0e614917f4937963bbf0ead963089 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:17.808973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:17.813066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:17.813066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:17.829445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:17.830863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:17.898829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1255/2544] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:17.909454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:17.910909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:17.944725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:17.946509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:17.952765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:17.974381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38299fc-c98e-4c3e-87b5-635cf8cc445b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:17.981168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:17.981168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:17.981168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:17.981544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:17.990902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:17.993023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:18.003790 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1290/2545] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:18.020944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57645f67-1802-49a4-a5fe-ab30b7f1743d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:18.026347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:18.026979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:18.027262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:18.027576 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:18.029309 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:18.045191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:18.048265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:18.059641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1256/2546] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:18.602383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:18.604265 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:18.618940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911dbb4-4507-4748-beac-fc727e35983d service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:18.624557 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:18.625280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:18.625631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:18.625976 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] PATH_INFO: `/v3/projects/90bf301b365a4f3e97f78a382607263f` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:18.627363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=90bf301b365a4f3e97f78a382607263f)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:18.631858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:18.756862 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90bf301b365a4f3e97f78a382607263f', 'request_id': 'req-41948991-f859-422a-b5e9-9d3ff8413baf'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:18.762836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90bf301b365a4f3e97f78a382607263f', 'request_id': 'req-41948991-f859-422a-b5e9-9d3ff8413baf'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:18.769338 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90bf301b365a4f3e97f78a382607263f', 'request_id': 'req-41948991-f859-422a-b5e9-9d3ff8413baf'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:18.852962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] The token cache is being invalidate because project 90bf301b365a4f3e97f78a382607263f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:18.853268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:18.853940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:18.855737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1291/2547] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:18 2022] DELETE /identity/v3/projects/90bf301b365a4f3e97f78a382607263f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:18.859788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:18.871232 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5bfd1a-f18f-4a25-b852-f533802d1af6 service placement] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:18.931198 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:18.931843 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:18.932127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:18.932402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] PATH_INFO: `/v3/users/35af368e680444c180d937bcc760567e` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:18.933522 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35af368e680444c180d937bcc760567e)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:18.935476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:19.149164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35af368e680444c180d937bcc760567e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:19.149986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35af368e680444c180d937bcc760567e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:19.150937 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35af368e680444c180d937bcc760567e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:19.286115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35af368e680444c180d937bcc760567e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:19.293747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e18a890c-aba0-4cfc-8866-246981f4a46c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35af368e680444c180d937bcc760567e'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:19.311554 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1257/2548] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:18 2022] DELETE /identity/v3/users/35af368e680444c180d937bcc760567e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:19.813874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:19.815869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:19.832599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41948991-f859-422a-b5e9-9d3ff8413baf admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:19.915000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:19.916122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:19.916567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:19.916988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] PATH_INFO: `/v3/projects/7be9df759ead4cab85eee385c24b9388` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:19.918372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be9df759ead4cab85eee385c24b9388)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:19.920518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:19.930680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:20.081539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be9df759ead4cab85eee385c24b9388', 'request_id': 'req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:20.088868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be9df759ead4cab85eee385c24b9388', 'request_id': 'req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:20.096887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be9df759ead4cab85eee385c24b9388', 'request_id': 'req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:20.188689 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] The token cache is being invalidate because project 7be9df759ead4cab85eee385c24b9388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:20.188982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:20.189611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4e03f9b1-93e6-4499-b430-2f4369e7ef7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:20.191155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1292/2549] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:19 2022] DELETE /identity/v3/projects/7be9df759ead4cab85eee385c24b9388 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:20.196822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f060084-8bfa-4e1a-96c4-d97efac6120d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.197297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f060084-8bfa-4e1a-96c4-d97efac6120d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.197636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f060084-8bfa-4e1a-96c4-d97efac6120d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.218096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5f060084-8bfa-4e1a-96c4-d97efac6120d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:20.219346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5f060084-8bfa-4e1a-96c4-d97efac6120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:20.225559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:20.302274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1258/2550] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:20.311559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052277-14e9-4fa4-b7ad-3c25869b0b78 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.314543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052277-14e9-4fa4-b7ad-3c25869b0b78 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.314543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052277-14e9-4fa4-b7ad-3c25869b0b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.343197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-06052277-14e9-4fa4-b7ad-3c25869b0b78 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:20.344876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-06052277-14e9-4fa4-b7ad-3c25869b0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:20.356292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1293/2551] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:20.360301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f060084-8bfa-4e1a-96c4-d97efac6120d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:20.373067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f060084-8bfa-4e1a-96c4-d97efac6120d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:20.378699 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:20.379416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.379761 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.380102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.380830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:20.381199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:20.383667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:20.396561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1259/2552] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:11:20.401822 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06052277-14e9-4fa4-b7ad-3c25869b0b78 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:20.412519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06052277-14e9-4fa4-b7ad-3c25869b0b78 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:20.418174 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:20.418848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.419135 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.419413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.420048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:20.422264 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:20.523427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1294/2553] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:20 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:20.527883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:20.542579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb98c0f-965d-4e7e-b3f6-2658d96be5c2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:20.612545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:20.613088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.613373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.613647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.614894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:20.617950 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:20.696277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1260/2554] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:20 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:20.697577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:20.708167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58283268-47b2-4ae0-8c96-05b5099c404a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:20.712956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:20.713675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.714021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.714331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.714973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:20.716574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:20.727462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1295/2555] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:20.731493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:20.743492 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21f4a7-1ad6-44a9-85f2-0692781fc84b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:20.748775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48380c24-53ab-4051-bda2-89f35a1a6874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:20.749443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48380c24-53ab-4051-bda2-89f35a1a6874 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.749726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48380c24-53ab-4051-bda2-89f35a1a6874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.750002 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48380c24-53ab-4051-bda2-89f35a1a6874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.750811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48380c24-53ab-4051-bda2-89f35a1a6874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:20.752432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48380c24-53ab-4051-bda2-89f35a1a6874 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:20.764986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1261/2556] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:20.770421 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:20.779092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f88c5-35e2-4be9-a818-9c356d0770d3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:20.783804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:20.784466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.784742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.785014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] PATH_INFO: `/v3/projects/44e8ad2d4f7d4e93b0140377740a042a/users/53207c91b80346c5b4f820f0a30a3203/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.800896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e8ad2d4f7d4e93b0140377740a042a, user_id=53207c91b80346c5b4f820f0a30a3203, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:20.803563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:20.891904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1296/2557] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:20 2022] PUT /identity/v3/projects/44e8ad2d4f7d4e93b0140377740a042a/users/53207c91b80346c5b4f820f0a30a3203/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:20.893521 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805611-4576-4214-af91-33bab6075eac None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:20.893835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805611-4576-4214-af91-33bab6075eac None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:20.894108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13805611-4576-4214-af91-33bab6075eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:20.930674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-13805611-4576-4214-af91-33bab6075eac None None] MFA Rules not processed for user `53207c91b80346c5b4f820f0a30a3203`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:20.932153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-13805611-4576-4214-af91-33bab6075eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:21.019230 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1262/2558] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:21.031756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:21.033520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:21.107221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896b51a4-4bf8-4a92-b5fe-0f461c774608 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:21.111038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:21.111641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:21.111921 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:21.112188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:21.121538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:21.125027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:21.135671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13805611-4576-4214-af91-33bab6075eac None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:21.137400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-13805611-4576-4214-af91-33bab6075eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:21.139375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1297/2559] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:21.148558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:21.211311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13805611-4576-4214-af91-33bab6075eac None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:21.216108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:21.216864 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:21.217154 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:21.217586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:21.220028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:21.309646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:21.311157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:21.329056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1263/2560] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:22.470724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:22.472239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:22.487341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bedf0-6151-435f-8c49-534e1891590e service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:22.499280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:22.499280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:22.499280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:22.499280 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] PATH_INFO: `/v3/users/ce5e17020ce0436ca26936c73c4d4514` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:22.499675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5e17020ce0436ca26936c73c4d4514)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:22.499675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:22.670220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:22.687345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5e17020ce0436ca26936c73c4d4514'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:22.688027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5e17020ce0436ca26936c73c4d4514'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:22.688719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5e17020ce0436ca26936c73c4d4514'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:22.697413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5e17020ce0436ca26936c73c4d4514'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:22.777578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5e17020ce0436ca26936c73c4d4514'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:22.795826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1298/2561] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:22 2022] DELETE /identity/v3/users/ce5e17020ce0436ca26936c73c4d4514 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:23.043440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:23.054612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b4ee3-caa9-420d-9d66-9d6ecd7f064e service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:23.121058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:23.121965 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:23.122470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:23.122893 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] PATH_INFO: `/v3/projects/e7e01f19882d4b6eb2498d59aa8ad78f` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:23.124431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e01f19882d4b6eb2498d59aa8ad78f)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:23.127196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:23.250662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e01f19882d4b6eb2498d59aa8ad78f', 'request_id': 'req-fd78109f-07a7-47f6-a371-1fcc63b2990a'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.257928 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e01f19882d4b6eb2498d59aa8ad78f', 'request_id': 'req-fd78109f-07a7-47f6-a371-1fcc63b2990a'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.259473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:23.265748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e01f19882d4b6eb2498d59aa8ad78f', 'request_id': 'req-fd78109f-07a7-47f6-a371-1fcc63b2990a'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.371949 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] The token cache is being invalidate because project e7e01f19882d4b6eb2498d59aa8ad78f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:23.372255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.373061 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.378525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1264/2562] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:23 2022] DELETE /identity/v3/projects/e7e01f19882d4b6eb2498d59aa8ad78f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:23.379254 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:23.381021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:23.405414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a872ea-08a8-4f31-8cce-d22732d7b9fd admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:23.481628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:23.482516 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:23.482668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:23.482940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] PATH_INFO: `/v3/users/02b8d7dc4d084d44b298eb70eabfaee3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:23.484112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b8d7dc4d084d44b298eb70eabfaee3)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:23.486206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:23.699513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b8d7dc4d084d44b298eb70eabfaee3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.700511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b8d7dc4d084d44b298eb70eabfaee3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.701378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b8d7dc4d084d44b298eb70eabfaee3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.708946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b8d7dc4d084d44b298eb70eabfaee3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.805952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1e978702-8f89-4fe1-bd0c-970081d23357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b8d7dc4d084d44b298eb70eabfaee3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:23.825552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1299/2563] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:23 2022] DELETE /identity/v3/users/02b8d7dc4d084d44b298eb70eabfaee3 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:24.081115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:24.081115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:24.105752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78109f-07a7-47f6-a371-1fcc63b2990a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:24.190092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:24.190819 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.191054 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.191335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] PATH_INFO: `/v3/projects/8170a3d39ebc4e509d05ee721b3fb8b2` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.192831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8170a3d39ebc4e509d05ee721b3fb8b2)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:24.196727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:24.310166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8170a3d39ebc4e509d05ee721b3fb8b2', 'request_id': 'req-2ebbaf5d-c533-45f8-b26e-067cac045f92'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.315837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8170a3d39ebc4e509d05ee721b3fb8b2', 'request_id': 'req-2ebbaf5d-c533-45f8-b26e-067cac045f92'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.322962 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8170a3d39ebc4e509d05ee721b3fb8b2', 'request_id': 'req-2ebbaf5d-c533-45f8-b26e-067cac045f92'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.394790 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] The token cache is being invalidate because project 8170a3d39ebc4e509d05ee721b3fb8b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:24.395084 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.396317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.398491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1265/2564] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:24 2022] DELETE /identity/v3/projects/8170a3d39ebc4e509d05ee721b3fb8b2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:24.403356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186a53b-0c8c-4121-80fe-9ff51c07b5fe None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.403835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186a53b-0c8c-4121-80fe-9ff51c07b5fe None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.404143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186a53b-0c8c-4121-80fe-9ff51c07b5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.423965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7186a53b-0c8c-4121-80fe-9ff51c07b5fe None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:24.425186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7186a53b-0c8c-4121-80fe-9ff51c07b5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:24.451737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:24.468784 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:24.470734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:24.498208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1300/2565] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:24.510384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a13aff-6422-46e2-85c5-3cd4886aabd5 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.510680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a13aff-6422-46e2-85c5-3cd4886aabd5 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.512426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a13aff-6422-46e2-85c5-3cd4886aabd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.561719 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-15a13aff-6422-46e2-85c5-3cd4886aabd5 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:24.563331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-15a13aff-6422-46e2-85c5-3cd4886aabd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:24.567816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbaf5d-c533-45f8-b26e-067cac045f92 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:24.574850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1301/2566] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:24.577413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:24.578032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.578342 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.578618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] PATH_INFO: `/v3/users/53207c91b80346c5b4f820f0a30a3203` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.578734 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a13aff-6422-46e2-85c5-3cd4886aabd5 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:24.580010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53207c91b80346c5b4f820f0a30a3203)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:24.583597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:24.593863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a13aff-6422-46e2-85c5-3cd4886aabd5 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:24.599266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:24.599987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.600349 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.601258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.601988 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:24.602394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:24.604072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:24.617935 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1302/2567] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:11:24.622019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:24.624901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:24.633384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22a727-1301-4397-bec0-f2f9d69cd5ec admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:24.638259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:24.638903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.639183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.639457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.640083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:24.642146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:24.718129 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1303/2568] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:24 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:24.721996 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:24.790868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53207c91b80346c5b4f820f0a30a3203'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.791530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53207c91b80346c5b4f820f0a30a3203'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.792143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53207c91b80346c5b4f820f0a30a3203'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.815854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d552-56cd-4bcb-8625-a420adbeefae admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:24.821651 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:24.822497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.822928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.827559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.827559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:24.827559 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:24.905602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53207c91b80346c5b4f820f0a30a3203'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.922309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1304/2569] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:24 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:24.923329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-4c42bbf8-d5b2-4771-aff1-6014a0be8962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53207c91b80346c5b4f820f0a30a3203'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:24.924434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:24.936015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:24.946307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bcdb7-6f87-46fa-a0db-7a24688324e1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:24.951385 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:24.952117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.952397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.952670 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.953353 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:24.955061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:24.963525 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1266/2570] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:24 2022] DELETE /identity/v3/users/53207c91b80346c5b4f820f0a30a3203 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:24.965132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:24.965508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:24.965910 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:24.972865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1305/2571] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:24.977938 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:24.988975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb48816-f7f6-4b30-a1d1-e10ed13b371b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:24.995162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:24.997669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.086990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:25.086990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.087343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.087580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.088147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:25.089922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:25.094335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1267/2572] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:25.097816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:25.103645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:25.104207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1306/2573] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:25.105108 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.108071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:25.119289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbc070-af08-4658-82a6-827d8d523600 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:25.124256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:25.125121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.125624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.125764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] PATH_INFO: `/v3/projects/02c34172aa804fe695944f629a348c58/users/9ca8c5d1971a4ec1a448f2081ceb835b/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.156378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c34172aa804fe695944f629a348c58, user_id=9ca8c5d1971a4ec1a448f2081ceb835b, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:25.161883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:25.183267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fa2e2-103f-443e-b7f1-a11e6e0d75e6 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:25.187894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c2ba8-98d5-4717-96e4-e7a4c9453f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:25.187894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2ba8-98d5-4717-96e4-e7a4c9453f0c service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.188272 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2ba8-98d5-4717-96e4-e7a4c9453f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.188736 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c2ba8-98d5-4717-96e4-e7a4c9453f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.200862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c2ba8-98d5-4717-96e4-e7a4c9453f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:25.202471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c2ba8-98d5-4717-96e4-e7a4c9453f0c service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:25.217592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1268/2574] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:25.253535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1307/2575] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:25 2022] PUT /identity/v3/projects/02c34172aa804fe695944f629a348c58/users/9ca8c5d1971a4ec1a448f2081ceb835b/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:25.259569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.260098 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.260377 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.280268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:25.282599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.317207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:25.345710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1269/2576] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:25.360962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:25.362612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.382703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:25.386510 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.435298 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532fe45-915d-4c29-b378-076253589009 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:25.440499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:25.441500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.441500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.441917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.455710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:25.457305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:25.469727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1308/2577] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:25.486489 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec8918-93a2-49d9-8b2b-a3705129cd49 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:25.492235 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:25.492860 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.493226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.493674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.495474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.508150 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:25.510250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:25.520442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1270/2578] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:25.598896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:25.600817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.615043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760e806-6193-463b-a071-d4f206f7a134 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:25.621048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:25.624198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.624198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.624198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] PATH_INFO: `/v3/users/a1cdc3bfa3e046fa863131d7ff27d4a1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.624198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1cdc3bfa3e046fa863131d7ff27d4a1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:25.626044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:25.794106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1cdc3bfa3e046fa863131d7ff27d4a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:25.794808 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1cdc3bfa3e046fa863131d7ff27d4a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:25.795523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1cdc3bfa3e046fa863131d7ff27d4a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:25.802523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1cdc3bfa3e046fa863131d7ff27d4a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:25.907018 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:25.908540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:25.909013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a5f49fce-7366-444f-a417-240703f51081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1cdc3bfa3e046fa863131d7ff27d4a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:25.924074 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55836d-0085-4aa5-afa4-eef25975be24 service placement] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:25.930434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:25.931216 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:25.931673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:25.932140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] PATH_INFO: `/v3/projects/44e8ad2d4f7d4e93b0140377740a042a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:25.932293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1309/2579] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:25 2022] DELETE /identity/v3/users/a1cdc3bfa3e046fa863131d7ff27d4a1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:25.939160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e8ad2d4f7d4e93b0140377740a042a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:25.948229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.076260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e8ad2d4f7d4e93b0140377740a042a', 'request_id': 'req-19d84316-8f42-40d3-a890-bf435dfcefd8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.082152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e8ad2d4f7d4e93b0140377740a042a', 'request_id': 'req-19d84316-8f42-40d3-a890-bf435dfcefd8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.089413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e8ad2d4f7d4e93b0140377740a042a', 'request_id': 'req-19d84316-8f42-40d3-a890-bf435dfcefd8'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.194175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] The token cache is being invalidate because project 44e8ad2d4f7d4e93b0140377740a042a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:26.194870 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.195401 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.196915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1271/2580] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:25 2022] DELETE /identity/v3/projects/44e8ad2d4f7d4e93b0140377740a042a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:26.202148 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbf260-5a12-4c06-a19f-1d939dfd13bb None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.202462 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbf260-5a12-4c06-a19f-1d939dfd13bb None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.202740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbf260-5a12-4c06-a19f-1d939dfd13bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.228228 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-0cfbf260-5a12-4c06-a19f-1d939dfd13bb None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:26.229976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.232396 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:26.240334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0cfbf260-5a12-4c06-a19f-1d939dfd13bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:26.312412 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:26.325004 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:26.326637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d84316-8f42-40d3-a890-bf435dfcefd8 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.331453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.332093 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.332379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.332651 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] PATH_INFO: `/v3/projects/d809822ae9ba4c248d5941746efff780` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.333794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d809822ae9ba4c248d5941746efff780)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.336029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.340903 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1310/2581] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:26.348545 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df858e4-f789-4a13-bbba-792eba39e96a None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.348836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df858e4-f789-4a13-bbba-792eba39e96a None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.349112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df858e4-f789-4a13-bbba-792eba39e96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.370259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7df858e4-f789-4a13-bbba-792eba39e96a None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:26.371482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7df858e4-f789-4a13-bbba-792eba39e96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:26.379640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1311/2582] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:26.384253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df858e4-f789-4a13-bbba-792eba39e96a None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.393520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df858e4-f789-4a13-bbba-792eba39e96a None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.398830 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.399436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.399712 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.399982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.400630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:26.400931 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.402591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.414142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1312/2583] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:11:26.418329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.427794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088243e8-f88c-41f2-a025-d4c23e7eda76 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.433206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.433803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.438260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.438260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.438260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.438260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.475891 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd809822ae9ba4c248d5941746efff780', 'request_id': 'req-b93ea96f-8b99-42d1-8271-681351b563f5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.481934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd809822ae9ba4c248d5941746efff780', 'request_id': 'req-b93ea96f-8b99-42d1-8271-681351b563f5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.488134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd809822ae9ba4c248d5941746efff780', 'request_id': 'req-b93ea96f-8b99-42d1-8271-681351b563f5'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.543043 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1313/2584] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:26.544094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.561399 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] The token cache is being invalidate because project d809822ae9ba4c248d5941746efff780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:26.564833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.564833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-b93ea96f-8b99-42d1-8271-681351b563f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:26.564833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1272/2585] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:26 2022] DELETE /identity/v3/projects/d809822ae9ba4c248d5941746efff780 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:26.568612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596a74-7dc3-46e8-b694-14ad7bd59d6b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.568894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596a74-7dc3-46e8-b694-14ad7bd59d6b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.569166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596a74-7dc3-46e8-b694-14ad7bd59d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.589112 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ca596a74-7dc3-46e8-b694-14ad7bd59d6b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:26.590359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ca596a74-7dc3-46e8-b694-14ad7bd59d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:26.606433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294573e7-2bf3-4e87-b5e1-c7204fc893fe admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.611404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.612019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.612305 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.612583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.613226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.615949 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.653014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1273/2586] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:26.660653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f398-4acd-40b0-847e-b99b92366ef0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.660941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f398-4acd-40b0-847e-b99b92366ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.661218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f398-4acd-40b0-847e-b99b92366ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.681896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5dd1f398-4acd-40b0-847e-b99b92366ef0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:26.683142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5dd1f398-4acd-40b0-847e-b99b92366ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:26.691237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1274/2587] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:26.695660 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f398-4acd-40b0-847e-b99b92366ef0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.704588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f398-4acd-40b0-847e-b99b92366ef0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.709388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.710014 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.710328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.710672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.711237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:26.711540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.713141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.729630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1275/2588] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:11:26.733983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.734120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1314/2589] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:26.736783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.739327 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:26.744375 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a65d8-d6d5-43e7-8456-6569d1d1d851 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.748960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.749590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.749872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.750147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.750818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.752307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930a070-d92d-4cfa-bc8a-ba1b91f59962 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.752724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.757513 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.758160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.762796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.762796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.762796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.762796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.771234 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1315/2590] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:26.775878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.785678 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace208d-cb1d-416d-8ea0-7eb764e8751a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.790304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.791450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.791450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.791450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.792015 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.793582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.804855 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1316/2591] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:11:26.808789 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.819110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c406c19-85a9-47a9-b6dc-ac2fed42b490 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.825224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90608ee-1dac-42be-81fd-fa40029189fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.825894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90608ee-1dac-42be-81fd-fa40029189fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.826171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90608ee-1dac-42be-81fd-fa40029189fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.826473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90608ee-1dac-42be-81fd-fa40029189fd admin admin] PATH_INFO: `/v3/projects/80f7e6e5a0194c3aa602534a6c10be17/users/498df0ec8d364e0cbef7ddb05c1e02e3/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.842984 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90608ee-1dac-42be-81fd-fa40029189fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f7e6e5a0194c3aa602534a6c10be17, user_id=498df0ec8d364e0cbef7ddb05c1e02e3, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.845677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90608ee-1dac-42be-81fd-fa40029189fd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.879883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1276/2592] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:26.880877 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:26.932899 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:26.953528 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c5236-2960-44aa-8ca2-e2aa07573ecf admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:26.958322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:26.959347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.959644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.960665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:26.961285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:26.963353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:26.968131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1317/2593] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:26 2022] PUT /identity/v3/projects/80f7e6e5a0194c3aa602534a6c10be17/users/498df0ec8d364e0cbef7ddb05c1e02e3/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:26.979183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:26.980484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:26.980861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:27.010578 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] MFA Rules not processed for user `498df0ec8d364e0cbef7ddb05c1e02e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:27.012002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:27.064915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:27.097948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1277/2594] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:27.100431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:27.102770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1318/2595] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:27.115112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:27.117773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:27.118711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a929f2-bd25-4dbf-a1b3-9654f86d71bf admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:27.187303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:27.193159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:27.193159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:27.193159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:27.193159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:27.193159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:27.196479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09b169-3f9f-46cf-bc09-b01936831c87 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:27.200876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990660ce5a9f4213a5b42b3824d0b567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:27.201485 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:27.201764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:27.202077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:27.206663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1278/2596] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:27.207637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:27.215705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:27.217768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:27.219404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb0594-1a23-481d-b2f7-62d0803014cc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:27.226177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e2f80-dd52-4882-a82f-caa653f38895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:27.226923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e2f80-dd52-4882-a82f-caa653f38895 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:27.227446 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e2f80-dd52-4882-a82f-caa653f38895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:27.228228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e2f80-dd52-4882-a82f-caa653f38895 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:27.228914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e2f80-dd52-4882-a82f-caa653f38895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:27.231010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e2f80-dd52-4882-a82f-caa653f38895 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:27.234368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1319/2597] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:27.244185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1279/2598] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:27.248868 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:27.268692 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba1f36-e7c5-4b81-8177-b55efa026828 service glance] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:27.270749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:27.271524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:27.272021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:27.272387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] PATH_INFO: `/v3/projects/89b78a516ab845659140d1bc70d567af/users/dd9fac43ae77477d9415306947bb19a9/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:27.294915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b78a516ab845659140d1bc70d567af, user_id=dd9fac43ae77477d9415306947bb19a9, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:27.298088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:27.373528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1320/2599] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:27 2022] PUT /identity/v3/projects/89b78a516ab845659140d1bc70d567af/users/dd9fac43ae77477d9415306947bb19a9/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:27.384301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482336f-1f80-40e3-ab2d-5093580e2b8e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:27.384588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482336f-1f80-40e3-ab2d-5093580e2b8e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:27.384863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482336f-1f80-40e3-ab2d-5093580e2b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:27.425426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b482336f-1f80-40e3-ab2d-5093580e2b8e None None] MFA Rules not processed for user `dd9fac43ae77477d9415306947bb19a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:27.427430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b482336f-1f80-40e3-ab2d-5093580e2b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:27.510611 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1280/2600] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:27.522183 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:27.524199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:27.620668 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b8e8-6205-417c-bf71-fb0f81c79df7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:27.626500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa90762-d75e-4a73-98eb-22912f05641b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:27.627111 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa90762-d75e-4a73-98eb-22912f05641b service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:27.627386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa90762-d75e-4a73-98eb-22912f05641b service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:27.627657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa90762-d75e-4a73-98eb-22912f05641b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:27.640392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa90762-d75e-4a73-98eb-22912f05641b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:27.643809 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa90762-d75e-4a73-98eb-22912f05641b service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:27.653876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1321/2601] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:31.151634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482336f-1f80-40e3-ab2d-5093580e2b8e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:31.214960 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:31.231526 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482336f-1f80-40e3-ab2d-5093580e2b8e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:31.236365 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:31.237251 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:31.242151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:31.242151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] PATH_INFO: `/v3/users/dd9fac43ae77477d9415306947bb19a9` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:31.242151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9fac43ae77477d9415306947bb19a9)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:31.242151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:31.434710 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9fac43ae77477d9415306947bb19a9'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:31.439641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9fac43ae77477d9415306947bb19a9'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:31.441861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9fac43ae77477d9415306947bb19a9'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:31.558999 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9fac43ae77477d9415306947bb19a9'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:31.592771 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9fac43ae77477d9415306947bb19a9'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:31.649144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1281/2602] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:31 2022] DELETE /identity/v3/users/dd9fac43ae77477d9415306947bb19a9 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:31.649460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc98a35-f84c-4794-acb6-0aae6e9721a9 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:31.649460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc98a35-f84c-4794-acb6-0aae6e9721a9 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:31.649460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc98a35-f84c-4794-acb6-0aae6e9721a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:31.677062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7fc98a35-f84c-4794-acb6-0aae6e9721a9 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:31.678396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7fc98a35-f84c-4794-acb6-0aae6e9721a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:31.738871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:31.766289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1322/2603] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:31.775885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:31.778714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:31.876339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672d46e-7b86-4474-955e-16ff2c65fc6e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:31.881123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:31.883419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:31.883718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:31.883992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:31.906440 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:31.910858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:31.931973 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1282/2604] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:32.055454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71359f71-9696-4c0e-a22e-eb8c7a70b100 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:32.055750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71359f71-9696-4c0e-a22e-eb8c7a70b100 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:32.056599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71359f71-9696-4c0e-a22e-eb8c7a70b100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:32.090077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-71359f71-9696-4c0e-a22e-eb8c7a70b100 None None] MFA Rules not processed for user `9ca8c5d1971a4ec1a448f2081ceb835b`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:32.092307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-71359f71-9696-4c0e-a22e-eb8c7a70b100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:32.170569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1323/2605] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:32.174975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:32.176482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:32.189922 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30fe98-006f-478b-bb24-129e9877c549 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:32.201990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:32.203121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:32.203121 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:32.203337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:32.204170 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:32.206583 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:32.277582 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1283/2606] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:32 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:32.281566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71359f71-9696-4c0e-a22e-eb8c7a70b100 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:32.311803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71359f71-9696-4c0e-a22e-eb8c7a70b100 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:32.374205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:32.376137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:32.425746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:32.426478 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:32.426865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:32.427295 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:32.428078 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:32.431169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:32.470600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:32.502859 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd45b4-a4f5-43f3-96d3-2f4539017458 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:32.515432 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:32.520643 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1324/2607] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:32 2022] POST /identity/v3/users => generated 296 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:32.521676 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:32.523079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:32.523448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:32.523733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] PATH_INFO: `/v3/projects/89b78a516ab845659140d1bc70d567af` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:32.530078 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b78a516ab845659140d1bc70d567af)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:32.533556 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:32.539102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353d1b2-b843-411c-92c2-8a7eb23d7abb admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:32.558406 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:32.559108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:32.559396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:32.559673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:32.560468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:32.562262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:32.588456 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1325/2608] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:32.592613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:32.608593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fbb95-8ce4-4acd-ad90-6748cc600db4 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:32.615156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:32.616928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:32.617524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:32.622268 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] PATH_INFO: `/v3/projects/77c4ea94394e48aeb13f7e5cc391f0ba/users/699570a9d2ab42da873fca852e4f6fd4/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:32.643279 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c4ea94394e48aeb13f7e5cc391f0ba, user_id=699570a9d2ab42da873fca852e4f6fd4, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:32.649056 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:32.740965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1326/2609] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:32 2022] PUT /identity/v3/projects/77c4ea94394e48aeb13f7e5cc391f0ba/users/699570a9d2ab42da873fca852e4f6fd4/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:32.745066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:32.757270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b78a516ab845659140d1bc70d567af', 'request_id': 'req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:32.766662 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b78a516ab845659140d1bc70d567af', 'request_id': 'req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:32.803458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b78a516ab845659140d1bc70d567af', 'request_id': 'req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:32.948206 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] The token cache is being invalidate because project 89b78a516ab845659140d1bc70d567af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:32.948794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:32.951608 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-45917f97-2ca9-44a4-8f7a-1d58ca79d0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:32.954153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1284/2610] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:32 2022] DELETE /identity/v3/projects/89b78a516ab845659140d1bc70d567af => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:32.960439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4899b7-1204-4091-93cf-5a132c646f2f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:32.963120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4899b7-1204-4091-93cf-5a132c646f2f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:32.963120 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4899b7-1204-4091-93cf-5a132c646f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.018805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:33.026905 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-1d4899b7-1204-4091-93cf-5a132c646f2f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:33.028254 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-1d4899b7-1204-4091-93cf-5a132c646f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:33.056157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664bd0-0e9c-49e9-84fb-56bd07924b1b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.061876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.064662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.064662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.064662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.064662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.065675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:33.078345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1327/2611] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:11:33.082747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:33.087313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:33.102133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf48d-28c7-46b0-b305-44ef3fa4482e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.108821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.110318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.111156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.111992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] PATH_INFO: `/v3/projects/77c4ea94394e48aeb13f7e5cc391f0ba/users/699570a9d2ab42da873fca852e4f6fd4/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.116975 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c4ea94394e48aeb13f7e5cc391f0ba, user_id=699570a9d2ab42da873fca852e4f6fd4, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.120449 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:33.144496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1285/2613] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:33.158430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1328/2613] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:33 2022] PUT /identity/v3/projects/77c4ea94394e48aeb13f7e5cc391f0ba/users/699570a9d2ab42da873fca852e4f6fd4/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:33.167861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3676e02-1d92-4638-856c-e6203ceb98b4 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.171133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3676e02-1d92-4638-856c-e6203ceb98b4 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.171496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3676e02-1d92-4638-856c-e6203ceb98b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.225004 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-b3676e02-1d92-4638-856c-e6203ceb98b4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:33.226328 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-b3676e02-1d92-4638-856c-e6203ceb98b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:33.304227 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1286/2614] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:33.309011 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:33.322694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5c612-ac08-46d7-baa6-57021d2ce7cd admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.328679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.329877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.330569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.330569 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.333065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:33.333065 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.333253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:33.360502 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1329/2615] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:11:33.364836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3676e02-1d92-4638-856c-e6203ceb98b4 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:33.377906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3676e02-1d92-4638-856c-e6203ceb98b4 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.385797 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.386531 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.388968 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.389142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.389807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.391914 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:33.409850 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:33.472700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1287/2616] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:33 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:33.478343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:33.493794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d4b86-5bde-485f-8758-6862bf7b57a2 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.629920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.630685 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.631553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.631553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.633071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.635262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:33.759774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1330/2617] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:33 2022] POST /identity/v3/users => generated 302 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:33.764529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:33.779223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257967c-d8d2-4cf0-81f3-182254ce490d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.785842 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.786494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.786770 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.787035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.800515 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.805338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:33.857414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1288/2618] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:33.870012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:33.895598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c8db-0296-49be-a03a-a8740f9485f3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.902274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5de8-919d-4323-992c-808fa759a2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.902917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5de8-919d-4323-992c-808fa759a2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.903199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5de8-919d-4323-992c-808fa759a2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.903474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5de8-919d-4323-992c-808fa759a2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.904617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5de8-919d-4323-992c-808fa759a2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.906554 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5de8-919d-4323-992c-808fa759a2b8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:33.918143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1331/2619] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:33.922706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:33.934714 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b98a04-e9de-4b99-87e9-18bfda880eab admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:33.939304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:33.946188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:33.946188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:33.946188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] PATH_INFO: `/v3/projects/e56ce96d8b064eb9b2adb82078335cea/users/3c03b6ce978a4964a2ddcbc9b3950f1c/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:33.959102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56ce96d8b064eb9b2adb82078335cea, user_id=3c03b6ce978a4964a2ddcbc9b3950f1c, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:33.962413 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:34.057382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1289/2620] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:33 2022] PUT /identity/v3/projects/e56ce96d8b064eb9b2adb82078335cea/users/3c03b6ce978a4964a2ddcbc9b3950f1c/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:34.062653 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf1e5b-9e81-4d09-b3ca-fc2af8ea13ac None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:34.062858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf1e5b-9e81-4d09-b3ca-fc2af8ea13ac None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:34.063182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf1e5b-9e81-4d09-b3ca-fc2af8ea13ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:34.090833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-cdaf1e5b-9e81-4d09-b3ca-fc2af8ea13ac None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:34.092677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-cdaf1e5b-9e81-4d09-b3ca-fc2af8ea13ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:34.185167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1332/2621] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:34.197482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:34.200051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:34.281550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d8a9d-4784-4110-bc36-ae4273242c72 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:34.305332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:34.306168 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:34.306499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:34.306849 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:34.338825 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:34.340434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:34.371218 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1290/2622] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:36.601752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c3e4a-d22a-40ed-b87a-a84f53da14e3 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:36.602285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c3e4a-d22a-40ed-b87a-a84f53da14e3 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:36.610317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c3e4a-d22a-40ed-b87a-a84f53da14e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:36.648985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c40c3e4a-d22a-40ed-b87a-a84f53da14e3 None None] MFA Rules not processed for user `3c03b6ce978a4964a2ddcbc9b3950f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:36.651514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c40c3e4a-d22a-40ed-b87a-a84f53da14e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:36.663350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:36.745785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1333/2623] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:36.751188 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:36.771005 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85076e-6e1f-4145-a726-7ff7072cfe6f service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:36.780568 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:36.781654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:36.782337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:36.782806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:36.783670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:36.785855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:36.885383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1291/2624] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:36 2022] POST /identity/v3/projects => generated 403 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:36.889452 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c3e4a-d22a-40ed-b87a-a84f53da14e3 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:36.906771 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c3e4a-d22a-40ed-b87a-a84f53da14e3 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:36.990536 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:36.991950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:36.992266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:36.992492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:36.993744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:36.995674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:37.074047 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1334/2625] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:36 2022] POST /identity/v3/users => generated 308 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:37.084580 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:37.095753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76dd35-0290-485a-a2c6-ae50c1fa6440 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:37.102532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:37.103552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:37.104067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:37.104718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:37.105341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:37.107744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:37.125262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1292/2626] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:37.128607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:37.141006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b6da-22f7-41c4-8983-19ea7b64f854 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:37.149095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:37.149824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:37.150051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:37.150377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] PATH_INFO: `/v3/projects/e6584cb94040476193b0cbed1c768e66/users/dcace66b66814c22b7a14707d61bec7b/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:37.177546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6584cb94040476193b0cbed1c768e66, user_id=dcace66b66814c22b7a14707d61bec7b, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:37.181351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:37.269086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1335/2627] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:37 2022] PUT /identity/v3/projects/e6584cb94040476193b0cbed1c768e66/users/dcace66b66814c22b7a14707d61bec7b/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:37.272501 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:37.291823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa54b41-0c81-4a17-91ee-4f73c4c7089d admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:37.392231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f957c8-70f0-4d6d-aaff-7483aa9acfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:37.392857 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f957c8-70f0-4d6d-aaff-7483aa9acfce admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:37.393210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f957c8-70f0-4d6d-aaff-7483aa9acfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:37.393384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f957c8-70f0-4d6d-aaff-7483aa9acfce admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:37.394027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f957c8-70f0-4d6d-aaff-7483aa9acfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:37.395717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f957c8-70f0-4d6d-aaff-7483aa9acfce admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:37.410076 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1293/2628] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:37.415390 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:37.426783 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcca9e-3b5f-4fcc-9ee2-ff842d065f63 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:37.431674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:37.432333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:37.432606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:37.432877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] PATH_INFO: `/v3/projects/e6584cb94040476193b0cbed1c768e66/users/dcace66b66814c22b7a14707d61bec7b/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:37.434824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6584cb94040476193b0cbed1c768e66, user_id=dcace66b66814c22b7a14707d61bec7b, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:37.437408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:37.449222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1336/2629] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:37 2022] PUT /identity/v3/projects/e6584cb94040476193b0cbed1c768e66/users/dcace66b66814c22b7a14707d61bec7b/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:39.058801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ec73e-34ae-4861-98a0-b5c4dddde113 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:39.059767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ec73e-34ae-4861-98a0-b5c4dddde113 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:39.060205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ec73e-34ae-4861-98a0-b5c4dddde113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:39.107590 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:39.112423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-903ec73e-34ae-4861-98a0-b5c4dddde113 None None] MFA Rules not processed for user `dcace66b66814c22b7a14707d61bec7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:39.112722 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:39.113867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-903ec73e-34ae-4861-98a0-b5c4dddde113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:39.140971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:39.213761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942c6e5-b16e-4246-bdf4-3e7bcc65c9d7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:39.218259 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:39.220044 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:39.220350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:39.220780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:39.226331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:39.251317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1294/2630] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:39.267933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:39.344543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:39.346006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:39.359814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1337/2631] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:39.908884 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032b6c-cd24-416f-b519-d55c9f3a8aa0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:39.909600 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032b6c-cd24-416f-b519-d55c9f3a8aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:39.909948 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032b6c-cd24-416f-b519-d55c9f3a8aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:39.965255 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d6032b6c-cd24-416f-b519-d55c9f3a8aa0 None None] MFA Rules not processed for user `699570a9d2ab42da873fca852e4f6fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:39.966542 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d6032b6c-cd24-416f-b519-d55c9f3a8aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:40.078444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1295/2632] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:40.095409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:40.097459 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:40.194389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f742964-be18-4bcc-a1cb-557e1a4fcdda service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:40.200416 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:40.201048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:40.201328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:40.201600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:40.203622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:40.280397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:40.282175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:40.308654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1338/2633] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:41.419618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6032b6c-cd24-416f-b519-d55c9f3a8aa0 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:41.437618 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6032b6c-cd24-416f-b519-d55c9f3a8aa0 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:41.445053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:41.445520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:41.445966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:41.446441 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] PATH_INFO: `/v3/users/3c03b6ce978a4964a2ddcbc9b3950f1c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:41.448006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c03b6ce978a4964a2ddcbc9b3950f1c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:41.450577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:41.696500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c03b6ce978a4964a2ddcbc9b3950f1c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:41.697632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c03b6ce978a4964a2ddcbc9b3950f1c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:41.698565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c03b6ce978a4964a2ddcbc9b3950f1c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:41.761297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c03b6ce978a4964a2ddcbc9b3950f1c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:41.770987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c03b6ce978a4964a2ddcbc9b3950f1c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:41.801534 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1296/2634] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:41 2022] DELETE /identity/v3/users/3c03b6ce978a4964a2ddcbc9b3950f1c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:42.092072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:42.112419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4ab91-3d8d-48ec-a83b-c0b1a4073570 service nova] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:42.183470 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:42.203861 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:42.205095 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:42.205572 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:42.206350 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] PATH_INFO: `/v3/projects/e56ce96d8b064eb9b2adb82078335cea` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:42.207895 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56ce96d8b064eb9b2adb82078335cea)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:42.211461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:42.340321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56ce96d8b064eb9b2adb82078335cea', 'request_id': 'req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.348662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56ce96d8b064eb9b2adb82078335cea', 'request_id': 'req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.361218 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56ce96d8b064eb9b2adb82078335cea', 'request_id': 'req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.468048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] The token cache is being invalidate because project e56ce96d8b064eb9b2adb82078335cea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:42.468436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.469216 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.475056 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1339/2635] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:42 2022] DELETE /identity/v3/projects/e56ce96d8b064eb9b2adb82078335cea => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:42.477029 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:42.479064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:42.496721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f68b1a-992e-45f4-a9bc-32d288be5fd6 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:42.508626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:42.623035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:42.631111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:42.631111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:42.631111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] PATH_INFO: `/v3/users/dcace66b66814c22b7a14707d61bec7b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:42.631111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcace66b66814c22b7a14707d61bec7b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:42.631111 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:42.865638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcace66b66814c22b7a14707d61bec7b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.871523 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcace66b66814c22b7a14707d61bec7b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.872293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcace66b66814c22b7a14707d61bec7b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.946871 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcace66b66814c22b7a14707d61bec7b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:42.958276 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-686dbaf1-8ffd-491c-a93e-05644f65d635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcace66b66814c22b7a14707d61bec7b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:43.000654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1297/2636] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:42 2022] DELETE /identity/v3/users/dcace66b66814c22b7a14707d61bec7b => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:43.348036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:43.349617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:43.365414 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c968-4ce7-4f16-b2fc-d0d028fd95f7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:43.465533 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:43.466341 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:43.466669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:43.467149 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] PATH_INFO: `/v3/projects/e6584cb94040476193b0cbed1c768e66` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:43.468441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6584cb94040476193b0cbed1c768e66)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:43.471690 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:43.637969 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6584cb94040476193b0cbed1c768e66', 'request_id': 'req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:43.645158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6584cb94040476193b0cbed1c768e66', 'request_id': 'req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:43.659800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6584cb94040476193b0cbed1c768e66', 'request_id': 'req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:43.772748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] The token cache is being invalidate because project e6584cb94040476193b0cbed1c768e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:43.773206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:43.776211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-90fc01a6-a3e0-4b4d-8e96-41314d40d20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:43.776211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1340/2637] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:43 2022] DELETE /identity/v3/projects/e6584cb94040476193b0cbed1c768e66 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:43.781085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101964-7ec8-4882-8847-cb5047ecb0a6 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:43.781634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101964-7ec8-4882-8847-cb5047ecb0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:43.781729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101964-7ec8-4882-8847-cb5047ecb0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:43.810921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2e101964-7ec8-4882-8847-cb5047ecb0a6 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:43.813394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2e101964-7ec8-4882-8847-cb5047ecb0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:43.955876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1298/2638] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:43.967146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f133c-624e-4b8f-86e9-47f39219f109 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:43.967505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f133c-624e-4b8f-86e9-47f39219f109 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:43.967848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f133c-624e-4b8f-86e9-47f39219f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.031765 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9b0f133c-624e-4b8f-86e9-47f39219f109 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:44.033156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9b0f133c-624e-4b8f-86e9-47f39219f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:44.067371 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1341/2639] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:44.073661 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e101964-7ec8-4882-8847-cb5047ecb0a6 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:44.090127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e101964-7ec8-4882-8847-cb5047ecb0a6 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:44.100351 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:44.104169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:44.104169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:44.112321 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.121986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:44.122348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:44.124027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:44.157064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1299/2640] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:11:44.162061 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f133c-624e-4b8f-86e9-47f39219f109 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:44.167321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:44.175167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f133c-624e-4b8f-86e9-47f39219f109 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:44.189525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:44.191124 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:44.191474 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:44.193666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.195024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:44.201217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:44.331510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1342/2641] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:44 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:44.336514 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:44.356946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd9cc1-5b39-4961-8772-85162fbdf000 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:44.379285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:44.480487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:44.484895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:44.484895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:44.484895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.484895 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:44.486640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:44.635325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1300/2642] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:44 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:44.639823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:44.654062 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00068aaa-68f2-45a9-94c1-72cae83b0314 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:44.678117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:44.680743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:44.681125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:44.681499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.682333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:44.684182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:44.707167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1343/2643] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:11:44.712394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:44.726586 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53e863-49bf-4dca-8be4-439fd7ac8e70 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:44.751571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:44.751571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:44.751571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:44.751571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.751958 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:44.753523 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:44.769943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1301/2644] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:44.786795 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:44.801701 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec532bfb-a6cd-49d0-b622-6f9e057cc316 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:44.808107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc2ef7-bc6a-4187-ba51-39f251b885f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:44.809133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2ef7-bc6a-4187-ba51-39f251b885f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:44.809415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2ef7-bc6a-4187-ba51-39f251b885f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:44.809732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc2ef7-bc6a-4187-ba51-39f251b885f0 admin admin] PATH_INFO: `/v3/projects/a1502759a48d449d9acb973161230a33/users/28998afe665d45c7816e3ded4d0e34fb/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.827263 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc2ef7-bc6a-4187-ba51-39f251b885f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1502759a48d449d9acb973161230a33, user_id=28998afe665d45c7816e3ded4d0e34fb, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:44.830433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc2ef7-bc6a-4187-ba51-39f251b885f0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:44.954426 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:44.967452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:44.979973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1344/2645] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:44 2022] PUT /identity/v3/projects/a1502759a48d449d9acb973161230a33/users/28998afe665d45c7816e3ded4d0e34fb/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:44.992023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb7dd1-1d1f-4c17-9a45-8cd21520c83f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:44.992483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb7dd1-1d1f-4c17-9a45-8cd21520c83f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:44.993098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb7dd1-1d1f-4c17-9a45-8cd21520c83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:44.995484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf20da8-34a9-4c61-aecd-dd37cb164d53 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.035807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8eeb7dd1-1d1f-4c17-9a45-8cd21520c83f None None] MFA Rules not processed for user `28998afe665d45c7816e3ded4d0e34fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:45.037593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8eeb7dd1-1d1f-4c17-9a45-8cd21520c83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:45.062358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.063271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.064487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.065153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] PATH_INFO: `/v3/users/d13b7df898df4d238ea290fab8edc4a7` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.066881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13b7df898df4d238ea290fab8edc4a7)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.071306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.124337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1345/2646] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:45.128787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb7dd1-1d1f-4c17-9a45-8cd21520c83f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:45.141307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb7dd1-1d1f-4c17-9a45-8cd21520c83f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.145898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.146555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.146793 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.147058 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.148001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.149898 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.265847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1346/2647] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:45 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:45.269863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:45.322140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13b7df898df4d238ea290fab8edc4a7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.322954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13b7df898df4d238ea290fab8edc4a7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.323743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13b7df898df4d238ea290fab8edc4a7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.336992 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525118a-e8bf-4bc4-a8a0-7cea258e362a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.341819 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.342493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.344407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.344407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.344407 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.345600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.383612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13b7df898df4d238ea290fab8edc4a7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.391258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13b7df898df4d238ea290fab8edc4a7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.409253 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1302/2648] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:44 2022] DELETE /identity/v3/users/d13b7df898df4d238ea290fab8edc4a7 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:45.465125 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1347/2649] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:45 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:45.469713 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:45.471453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:45.489381 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:45.549803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0fa4b-e8d0-452a-aa7f-66651e15fafa admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.555132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.555837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.556115 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.556386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.557028 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.558724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.569751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1303/2650] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:45.573641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:45.579156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:45.586342 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e694a6-bc39-430b-b55e-e273e902f30b admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.590954 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.591601 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.591874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.592142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] PATH_INFO: `/v3/projects/eeae3f49407f45abae2f5da33ef6abc8/users/76ea05ec4ec74c9a87ec38c7d6bad112/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.608662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeae3f49407f45abae2f5da33ef6abc8, user_id=76ea05ec4ec74c9a87ec38c7d6bad112, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.611510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.651456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:45.653606 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:45.673409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e555a1d-af90-4aaf-a5e0-024fba5f6404 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.680841 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.681599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.681789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.682196 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] PATH_INFO: `/v3/projects/629e8810bbcb4623bcd6b6aee8f7dbb9` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.685369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629e8810bbcb4623bcd6b6aee8f7dbb9)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.685470 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.700071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1348/2651] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:45 2022] PUT /identity/v3/projects/eeae3f49407f45abae2f5da33ef6abc8/users/76ea05ec4ec74c9a87ec38c7d6bad112/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:45.704039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:45.773534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e8b60-c44c-49ee-ac17-fd9f51c0f408 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.778440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.779053 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.779333 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.779606 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.780250 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.781990 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.793487 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629e8810bbcb4623bcd6b6aee8f7dbb9', 'request_id': 'req-327ce8e6-8004-4e0d-97b0-67f28a481c56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.794683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1349/2652] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:11:45.797916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:45.801982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629e8810bbcb4623bcd6b6aee8f7dbb9', 'request_id': 'req-327ce8e6-8004-4e0d-97b0-67f28a481c56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.812398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e95614-b805-4e97-9825-c49888e417a4 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:45.813184 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629e8810bbcb4623bcd6b6aee8f7dbb9', 'request_id': 'req-327ce8e6-8004-4e0d-97b0-67f28a481c56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.890882 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] The token cache is being invalidate because project 629e8810bbcb4623bcd6b6aee8f7dbb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:45.892133 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.892483 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-327ce8e6-8004-4e0d-97b0-67f28a481c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:45.895807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1304/2653] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:45 2022] DELETE /identity/v3/projects/629e8810bbcb4623bcd6b6aee8f7dbb9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:45.903238 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ca3dc-2487-4745-8649-ce1751b26797 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.903804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ca3dc-2487-4745-8649-ce1751b26797 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.904285 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ca3dc-2487-4745-8649-ce1751b26797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.904424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadc21a-75ed-41e7-a710-d704c2641a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:45.905253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadc21a-75ed-41e7-a710-d704c2641a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.905539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadc21a-75ed-41e7-a710-d704c2641a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.905812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadc21a-75ed-41e7-a710-d704c2641a8a admin admin] PATH_INFO: `/v3/projects/eeae3f49407f45abae2f5da33ef6abc8/users/76ea05ec4ec74c9a87ec38c7d6bad112/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.907836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadc21a-75ed-41e7-a710-d704c2641a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeae3f49407f45abae2f5da33ef6abc8, user_id=76ea05ec4ec74c9a87ec38c7d6bad112, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:45.911846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadc21a-75ed-41e7-a710-d704c2641a8a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:45.931128 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-345ca3dc-2487-4745-8649-ce1751b26797 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:45.931486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1350/2654] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:45 2022] PUT /identity/v3/projects/eeae3f49407f45abae2f5da33ef6abc8/users/76ea05ec4ec74c9a87ec38c7d6bad112/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:45.932603 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-345ca3dc-2487-4745-8649-ce1751b26797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:45.942732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:45.943197 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:45.943573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:45.981599 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] MFA Rules not processed for user `76ea05ec4ec74c9a87ec38c7d6bad112`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:45.983021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:46.030055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1305/2655] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:46.037595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da6ae5-1649-4989-8996-73dce829065d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.037875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da6ae5-1649-4989-8996-73dce829065d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.038394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da6ae5-1649-4989-8996-73dce829065d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.059079 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e6da6ae5-1649-4989-8996-73dce829065d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:46.060376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e6da6ae5-1649-4989-8996-73dce829065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:46.074987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1306/2656] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:46.079045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:46.081027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da6ae5-1649-4989-8996-73dce829065d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.091011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da6ae5-1649-4989-8996-73dce829065d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:46.094372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1351/2657] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:46.097269 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:46.097883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.099989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.099989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.099989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:46.099989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:46.102026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:46.111009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.112563 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:46.122769 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1307/2658] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:11:46.127359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.139837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4aa580-4fd2-4886-b343-765e817842c0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:46.148405 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:46.149398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.149749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.150085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.151346 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:46.153672 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:46.166416 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:46.189057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3456e-0071-411a-beb1-fa6352d79221 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:46.193108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:46.193757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.194060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.194962 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.205335 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:46.211549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:46.220477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1352/2659] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:46.258503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1308/2660] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:46 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:46.273042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.294600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf79bc8-2e6b-4fec-9610-b4e3b8d8a446 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:46.402304 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:46.402823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.403107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.403882 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.404524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:46.406467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:46.562497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1353/2661] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:46 2022] POST /identity/v3/users => generated 288 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:46.566711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.578918 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8dc52-a875-4a25-abb0-d6dd84da9bad admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:46.587289 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:46.588415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.588720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.589165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.590187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:46.592733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:46.609715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1309/2662] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:11:46.613737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.624853 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfccab9-92b3-45e0-9ba5-81c2fdae49e0 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:46.630812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e97fab-7d42-42f8-b08e-88e2f5aed42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:46.631448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97fab-7d42-42f8-b08e-88e2f5aed42e admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.632515 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97fab-7d42-42f8-b08e-88e2f5aed42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.632906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e97fab-7d42-42f8-b08e-88e2f5aed42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.633775 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e97fab-7d42-42f8-b08e-88e2f5aed42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:46.636099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e97fab-7d42-42f8-b08e-88e2f5aed42e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:46.653394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1354/2663] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:46.658826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.670199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8deea9-3a5a-4313-bc98-c01f3729ae70 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:46.675393 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:46.675718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.676003 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.676337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] PATH_INFO: `/v3/projects/6a293d557d674e8d97596857b8d179b3/users/2f8117e04a1d4f7aa802df3dda87606e/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.700397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a293d557d674e8d97596857b8d179b3, user_id=2f8117e04a1d4f7aa802df3dda87606e, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:46.703209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:46.814486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1310/2664] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:46 2022] PUT /identity/v3/projects/6a293d557d674e8d97596857b8d179b3/users/2f8117e04a1d4f7aa802df3dda87606e/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:46.815810 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:46.816262 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:46.816471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:46.841740 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:46.843630 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:46.911468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1355/2665] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:46.925160 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:46.930730 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:46.999572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e614eb-4e10-4572-bb6d-8fba6b7e0fc2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:47.005211 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bd5403-5f40-42d4-b3fb-1d7ab8c67cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:47.006056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd5403-5f40-42d4-b3fb-1d7ab8c67cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:47.006386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd5403-5f40-42d4-b3fb-1d7ab8c67cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:47.006955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bd5403-5f40-42d4-b3fb-1d7ab8c67cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:47.018176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd5403-5f40-42d4-b3fb-1d7ab8c67cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:47.019645 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bd5403-5f40-42d4-b3fb-1d7ab8c67cc1 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:47.031587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1311/2666] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:47.315374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:47.317366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:47.330054 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a607ca-a5ec-401d-8260-bfbdd3a75c9d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:47.335607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:47.336357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:47.336687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:47.336965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] PATH_INFO: `/v3/users/28998afe665d45c7816e3ded4d0e34fb` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:47.338115 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28998afe665d45c7816e3ded4d0e34fb)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:47.340202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:47.354396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:47.663846 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28998afe665d45c7816e3ded4d0e34fb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:47.664922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28998afe665d45c7816e3ded4d0e34fb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:47.665900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28998afe665d45c7816e3ded4d0e34fb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:47.674255 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28998afe665d45c7816e3ded4d0e34fb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:47.794540 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28998afe665d45c7816e3ded4d0e34fb'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:47.817373 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1356/2667] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:47 2022] DELETE /identity/v3/users/28998afe665d45c7816e3ded4d0e34fb => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:47.829477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:47.829760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:47.830030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:47.851226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:47.852722 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:47.935779 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1312/2668] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:47.947274 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:47.950764 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:48.017209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c419a4-8e8f-445f-8923-a6d4e0a5ec43 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:48.021852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:48.023113 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:48.023397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:48.023869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:48.033706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:48.037099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:48.047380 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1357/2669] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:48.679371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:48.682072 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:48.695224 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8ba25-f14d-4a5b-a203-b9b103e43522 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:48.700912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:48.701824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:48.702577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:48.703060 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] PATH_INFO: `/v3/projects/a1502759a48d449d9acb973161230a33` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:48.706811 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1502759a48d449d9acb973161230a33)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:48.709119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:48.874834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1502759a48d449d9acb973161230a33', 'request_id': 'req-765f6b10-e9cf-448e-b900-050b7f5544af'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:48.879428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:48.882095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1502759a48d449d9acb973161230a33', 'request_id': 'req-765f6b10-e9cf-448e-b900-050b7f5544af'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:48.889369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1502759a48d449d9acb973161230a33', 'request_id': 'req-765f6b10-e9cf-448e-b900-050b7f5544af'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.001340 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] The token cache is being invalidate because project a1502759a48d449d9acb973161230a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:49.004322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.004322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.005125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1313/2670] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:48 2022] DELETE /identity/v3/projects/a1502759a48d449d9acb973161230a33 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:49.009728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:49.024742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bd30f-c8f9-410d-9d2d-f033d79a3fa6 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:49.089134 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:49.090000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:49.090435 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:49.090806 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] PATH_INFO: `/v3/users/76ea05ec4ec74c9a87ec38c7d6bad112` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:49.092060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ea05ec4ec74c9a87ec38c7d6bad112)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:49.094092 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:49.260794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:49.274207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ea05ec4ec74c9a87ec38c7d6bad112'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.274910 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ea05ec4ec74c9a87ec38c7d6bad112'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.275511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ea05ec4ec74c9a87ec38c7d6bad112'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.283181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ea05ec4ec74c9a87ec38c7d6bad112'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.372811 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-d37f0311-b7d2-419e-b012-2ac54640fdd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ea05ec4ec74c9a87ec38c7d6bad112'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:49.400321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1358/2671] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:49 2022] DELETE /identity/v3/users/76ea05ec4ec74c9a87ec38c7d6bad112 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:49.937145 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:49.937627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:49.955404 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f6b10-e9cf-448e-b900-050b7f5544af admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:50.027350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:50.028158 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.028443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.028721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] PATH_INFO: `/v3/projects/eeae3f49407f45abae2f5da33ef6abc8` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.029915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeae3f49407f45abae2f5da33ef6abc8)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:50.031906 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:50.151837 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeae3f49407f45abae2f5da33ef6abc8', 'request_id': 'req-86860a20-b8ea-4872-9407-2d34a5255af1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:50.160992 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeae3f49407f45abae2f5da33ef6abc8', 'request_id': 'req-86860a20-b8ea-4872-9407-2d34a5255af1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:50.168670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeae3f49407f45abae2f5da33ef6abc8', 'request_id': 'req-86860a20-b8ea-4872-9407-2d34a5255af1'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:50.272541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] The token cache is being invalidate because project eeae3f49407f45abae2f5da33ef6abc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:50.272541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:50.272541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-86860a20-b8ea-4872-9407-2d34a5255af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:50.276398 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1314/2672] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:49 2022] DELETE /identity/v3/projects/eeae3f49407f45abae2f5da33ef6abc8 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:50.281057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57af79-e115-4951-bf11-d23b8af00706 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.281539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57af79-e115-4951-bf11-d23b8af00706 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.281639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57af79-e115-4951-bf11-d23b8af00706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.304337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7d57af79-e115-4951-bf11-d23b8af00706 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:50.305683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7d57af79-e115-4951-bf11-d23b8af00706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:50.381359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1359/2673] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:50.389348 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b24d7-db82-4b03-85b4-19ffab052c78 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.389638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b24d7-db82-4b03-85b4-19ffab052c78 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.389921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b24d7-db82-4b03-85b4-19ffab052c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.411418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9a1b24d7-db82-4b03-85b4-19ffab052c78 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:50.413558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9a1b24d7-db82-4b03-85b4-19ffab052c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:50.425437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1315/2674] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:50.427261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57af79-e115-4951-bf11-d23b8af00706 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:50.438256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57af79-e115-4951-bf11-d23b8af00706 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:50.445840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:50.445840 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.446121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.446394 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.447433 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:50.447733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:50.449404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:50.469034 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1360/2675] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:11:50.473827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b24d7-db82-4b03-85b4-19ffab052c78 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:50.494209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b24d7-db82-4b03-85b4-19ffab052c78 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:50.501775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:50.502435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.502717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.502990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.503619 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:50.505665 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:50.601294 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1316/2676] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:50 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:50.603688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:50.620069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127a450-fb2b-42d5-9756-3a15b0fe890a admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:50.765499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:50.778442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.778442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.778442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.778442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:50.778442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:50.869527 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1361/2677] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:50 2022] POST /identity/v3/users => generated 285 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:50.873732 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:50.885107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65594a4f-7b63-4446-b75b-41800a4c087a admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:50.889838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:50.890630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.890986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.891358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.892223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:50.893956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:50.909361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1317/2678] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 22 11:11:50.911105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:50.923016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28b94-2082-441e-a884-f2488dbef038 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:50.928202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff985949-0f61-4536-9fae-63002a6657a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:50.928932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985949-0f61-4536-9fae-63002a6657a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.929553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985949-0f61-4536-9fae-63002a6657a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.929553 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff985949-0f61-4536-9fae-63002a6657a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:50.930114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff985949-0f61-4536-9fae-63002a6657a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:50.932098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff985949-0f61-4536-9fae-63002a6657a0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:50.946309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1362/2679] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:50.950868 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:50.968317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9790e-8ab7-4d01-a45e-6b29b21ef1a5 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:50.978376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:50.979140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:50.979496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:50.979834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] PATH_INFO: `/v3/projects/03b6384e3a4144e8a2971f7c21f2db49/users/cdfc7cf3dc1c41e4888e4d6204e23272/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.002126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b6384e3a4144e8a2971f7c21f2db49, user_id=cdfc7cf3dc1c41e4888e4d6204e23272, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:51.005085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:51.079162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1318/2680] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:50 2022] PUT /identity/v3/projects/03b6384e3a4144e8a2971f7c21f2db49/users/cdfc7cf3dc1c41e4888e4d6204e23272/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:51.094526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d89ea-5091-40ae-bad8-db5dfb59affd None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.095029 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d89ea-5091-40ae-bad8-db5dfb59affd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.095219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d89ea-5091-40ae-bad8-db5dfb59affd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.128473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-c37d89ea-5091-40ae-bad8-db5dfb59affd None None] MFA Rules not processed for user `cdfc7cf3dc1c41e4888e4d6204e23272`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:51.129746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c37d89ea-5091-40ae-bad8-db5dfb59affd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:51.134951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:51.213858 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1363/2681] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:51.218419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:51.230827 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:51.305990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6ada0-1f7b-4a92-885b-07763ff3eea6 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:51.312052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:51.312673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.313013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.313335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.313977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:51.316140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:51.411523 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1319/2682] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:51 2022] POST /identity/v3/projects => generated 369 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:51.413389 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d89ea-5091-40ae-bad8-db5dfb59affd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:51.424487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d89ea-5091-40ae-bad8-db5dfb59affd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:51.494120 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:51.495042 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.495163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.495375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.496109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:51.498045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:51.599114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1364/2683] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:51 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:51.600824 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:51.615821 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5dbdc8-90d8-4a7d-8053-0953cb73988b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:51.621319 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:51.622082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.622310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.622792 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.623301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:51.624933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:51.642337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1320/2684] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:51.646190 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:51.659357 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63008a07-062d-4388-adfe-fdcfab349e79 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:51.664657 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:51.666345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.666345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.666345 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] PATH_INFO: `/v3/projects/5f6fae2ca3f148a7985a3208fdbc5d82/users/8abec6d153fd44968fb176a972fe6a96/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.689484 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6fae2ca3f148a7985a3208fdbc5d82, user_id=8abec6d153fd44968fb176a972fe6a96, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:51.692322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:51.790166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1365/2685] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:51 2022] PUT /identity/v3/projects/5f6fae2ca3f148a7985a3208fdbc5d82/users/8abec6d153fd44968fb176a972fe6a96/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:51.798961 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:51.810592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b80de-2603-41d6-b9c1-63a43d86ba50 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:51.883855 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca16e7-1e53-47c9-9e7c-2dbfa1626778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:51.884507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca16e7-1e53-47c9-9e7c-2dbfa1626778 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.884788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca16e7-1e53-47c9-9e7c-2dbfa1626778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.885323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca16e7-1e53-47c9-9e7c-2dbfa1626778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.885716 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca16e7-1e53-47c9-9e7c-2dbfa1626778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:51.887876 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca16e7-1e53-47c9-9e7c-2dbfa1626778 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:51.901464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1321/2686] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:51.905749 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:51.918596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc38e5-4ed5-4d80-bd13-63448abffb00 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:51.928009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:51.929089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.929466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.929768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] PATH_INFO: `/v3/projects/5f6fae2ca3f148a7985a3208fdbc5d82/users/8abec6d153fd44968fb176a972fe6a96/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.931842 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6fae2ca3f148a7985a3208fdbc5d82, user_id=8abec6d153fd44968fb176a972fe6a96, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:51.934588 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:51.946500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1366/2687] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:51 2022] PUT /identity/v3/projects/5f6fae2ca3f148a7985a3208fdbc5d82/users/8abec6d153fd44968fb176a972fe6a96/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:51.959136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771fe5a-bef7-427f-b040-63561b911ebe None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:51.961258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771fe5a-bef7-427f-b040-63561b911ebe None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:51.961258 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771fe5a-bef7-427f-b040-63561b911ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:51.992491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-a771fe5a-bef7-427f-b040-63561b911ebe None None] MFA Rules not processed for user `8abec6d153fd44968fb176a972fe6a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:51.997447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a771fe5a-bef7-427f-b040-63561b911ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:52.071226 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1322/2688] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:52.082422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:52.084161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:52.146856 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274f33b-ae06-48f3-8bcb-8327fe74200c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:52.154137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:52.154807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:52.155089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:52.155488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:52.166271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:52.168178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:52.174951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:52.182005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1367/2689] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:53.356305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126923a-d191-4c5b-9b92-3c19646f1b7d None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:53.356305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126923a-d191-4c5b-9b92-3c19646f1b7d None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:53.356305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126923a-d191-4c5b-9b92-3c19646f1b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:53.362673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:53.386736 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f126923a-d191-4c5b-9b92-3c19646f1b7d None None] MFA Rules not processed for user `2f8117e04a1d4f7aa802df3dda87606e`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:53.392944 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f126923a-d191-4c5b-9b92-3c19646f1b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:53.493610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1323/2690] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:53.516186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:53.517906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:53.609639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3711999-79a1-4784-a474-81cdc5a1e49e service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:53.618640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6b855-c15b-4d55-be83-d65add27c550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:53.619277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6b855-c15b-4d55-be83-d65add27c550 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:53.619556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6b855-c15b-4d55-be83-d65add27c550 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:53.620040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6b855-c15b-4d55-be83-d65add27c550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:53.629155 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6b855-c15b-4d55-be83-d65add27c550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:53.630782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6b855-c15b-4d55-be83-d65add27c550 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:53.640518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1368/2691] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:55.696383 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126923a-d191-4c5b-9b92-3c19646f1b7d None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:55.776434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126923a-d191-4c5b-9b92-3c19646f1b7d None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:55.780959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:55.781640 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:55.781883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:55.782144 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] PATH_INFO: `/v3/users/cdfc7cf3dc1c41e4888e4d6204e23272` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:55.784647 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfc7cf3dc1c41e4888e4d6204e23272)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:55.785256 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:55.950657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfc7cf3dc1c41e4888e4d6204e23272'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:55.951938 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfc7cf3dc1c41e4888e4d6204e23272'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:55.953919 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfc7cf3dc1c41e4888e4d6204e23272'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:56.006930 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfc7cf3dc1c41e4888e4d6204e23272'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:56.018612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfc7cf3dc1c41e4888e4d6204e23272'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:56.039176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1324/2692] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:55 2022] DELETE /identity/v3/users/cdfc7cf3dc1c41e4888e4d6204e23272 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:56.044526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7c388-9115-4e7b-adf3-198423e0065f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:56.045039 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7c388-9115-4e7b-adf3-198423e0065f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:56.045171 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7c388-9115-4e7b-adf3-198423e0065f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:56.065637 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-84b7c388-9115-4e7b-adf3-198423e0065f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:56.066779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-84b7c388-9115-4e7b-adf3-198423e0065f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:56.145087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1369/2693] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:56.147129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:56.148594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:56.168712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:56.217658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a32af-5af4-4ded-9581-b25f96b061bf admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:56.221278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:56.221901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:56.222183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:56.222488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:56.234590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:56.236654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:56.244384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:56.244725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:56.245023 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:56.258577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1325/2694] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:56.269695 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] MFA Rules not processed for user `2f8117e04a1d4f7aa802df3dda87606e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:56.271206 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:56.360617 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1370/2695] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:57.054195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:57.064015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fbca5-bdf2-4d76-be38-b56f05abdb07 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:57.073532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:57.076013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:57.076301 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:57.078694 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:57.084573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:57.086636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:57.086852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:57.101707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4684c3-e994-410b-902d-f951c19a050d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:57.109020 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:57.109669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:57.109883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:57.110031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:57.110251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] PATH_INFO: `/v3/projects/03b6384e3a4144e8a2971f7c21f2db49` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:57.111396 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b6384e3a4144e8a2971f7c21f2db49)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:57.111632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:57.113284 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:57.126545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1326/2696] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:11:57.253664 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:57.273297 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b6384e3a4144e8a2971f7c21f2db49', 'request_id': 'req-67412b9f-0a61-4fc3-bb24-5828ebd6f979'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.281244 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b6384e3a4144e8a2971f7c21f2db49', 'request_id': 'req-67412b9f-0a61-4fc3-bb24-5828ebd6f979'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.287702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b6384e3a4144e8a2971f7c21f2db49', 'request_id': 'req-67412b9f-0a61-4fc3-bb24-5828ebd6f979'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.378212 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] The token cache is being invalidate because project 03b6384e3a4144e8a2971f7c21f2db49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:57.380763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.380763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.384848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1371/2697] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:57 2022] DELETE /identity/v3/projects/03b6384e3a4144e8a2971f7c21f2db49 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:57.385894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:57.398793 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ffe52-d0cb-42e2-a9f6-150b34be4afc service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:57.468114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:57.468852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:57.469134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:57.469408 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] PATH_INFO: `/v3/users/8abec6d153fd44968fb176a972fe6a96` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:57.470622 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8abec6d153fd44968fb176a972fe6a96)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:57.474208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:57.849553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8abec6d153fd44968fb176a972fe6a96'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.850282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8abec6d153fd44968fb176a972fe6a96'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.851504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8abec6d153fd44968fb176a972fe6a96'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.925195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8abec6d153fd44968fb176a972fe6a96'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.933390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8abec6d153fd44968fb176a972fe6a96'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:57.952427 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1327/2698] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:57 2022] DELETE /identity/v3/users/8abec6d153fd44968fb176a972fe6a96 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:58.234930 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:58.236458 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:58.261169 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67412b9f-0a61-4fc3-bb24-5828ebd6f979 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:58.347836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:58.348480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.348743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.349016 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] PATH_INFO: `/v3/users/498df0ec8d364e0cbef7ddb05c1e02e3` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.350162 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=498df0ec8d364e0cbef7ddb05c1e02e3)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:58.352220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:58.452722 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:58.454591 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:58.481047 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3740144-3241-48e1-a076-6931d88361b6 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:58.485981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:58.487915 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.488194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.488458 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] PATH_INFO: `/v3/projects/5f6fae2ca3f148a7985a3208fdbc5d82` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.489569 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f6fae2ca3f148a7985a3208fdbc5d82)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:58.492056 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:58.510507 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:58.596650 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498df0ec8d364e0cbef7ddb05c1e02e3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.597378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498df0ec8d364e0cbef7ddb05c1e02e3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.598368 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498df0ec8d364e0cbef7ddb05c1e02e3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.609454 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498df0ec8d364e0cbef7ddb05c1e02e3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.611153 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f6fae2ca3f148a7985a3208fdbc5d82', 'request_id': 'req-50af3fbf-1d2a-4e6e-81b9-5411e6206749'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.618570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f6fae2ca3f148a7985a3208fdbc5d82', 'request_id': 'req-50af3fbf-1d2a-4e6e-81b9-5411e6206749'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.625288 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f6fae2ca3f148a7985a3208fdbc5d82', 'request_id': 'req-50af3fbf-1d2a-4e6e-81b9-5411e6206749'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.663788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8caa1b34-52ba-418e-9d7b-e2830cf73fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498df0ec8d364e0cbef7ddb05c1e02e3'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.683258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1372/2699] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:11:58 2022] DELETE /identity/v3/users/498df0ec8d364e0cbef7ddb05c1e02e3 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:11:58.687677 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.688180 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.688487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.716857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:58.716857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:58.741998 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] The token cache is being invalidate because project 5f6fae2ca3f148a7985a3208fdbc5d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:58.742326 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.742966 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-50af3fbf-1d2a-4e6e-81b9-5411e6206749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:58.744435 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1328/2700] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:58 2022] DELETE /identity/v3/projects/5f6fae2ca3f148a7985a3208fdbc5d82 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:58.748419 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:58.749369 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4b3ed-e7b9-4b28-9e95-3f5f6390627b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.749717 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4b3ed-e7b9-4b28-9e95-3f5f6390627b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.750017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4b3ed-e7b9-4b28-9e95-3f5f6390627b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.770241 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-dcc4b3ed-e7b9-4b28-9e95-3f5f6390627b None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:58.771747 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-dcc4b3ed-e7b9-4b28-9e95-3f5f6390627b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:58.781543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1373/2701] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:58.797114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:58.798947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:58.833657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1329/2702] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:58.841989 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644576a-6993-4aa5-a8d4-fe0e20862a9e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.842310 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644576a-6993-4aa5-a8d4-fe0e20862a9e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.842589 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644576a-6993-4aa5-a8d4-fe0e20862a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.861957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4644576a-6993-4aa5-a8d4-fe0e20862a9e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:58.863195 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4644576a-6993-4aa5-a8d4-fe0e20862a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:58.863728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1548e6-f637-4113-8721-7d06f9c318d2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:58.867612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:58.868221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.868503 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.868956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.871578 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1330/2703] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:11:58.875777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644576a-6993-4aa5-a8d4-fe0e20862a9e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:58.878892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:58.883392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:58.885451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644576a-6993-4aa5-a8d4-fe0e20862a9e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:58.890437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:58.890997 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.891278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.891501 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1374/2704] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:11:58.891615 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.892183 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:11:58.892480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:58.894077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:58.906353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1331/2705] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 22 11:11:58.910618 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:58.920762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0844dcf-20dd-418c-8a46-334246b96701 service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:58.925746 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:58.926393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:58.926680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:58.926959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:58.927598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:58.930743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.073375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1375/2706] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:58 2022] POST /identity/v3/projects => generated 393 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:59.073646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.088783 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330924e6-c66c-456a-b43e-c196c0bd6882 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.155715 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.156439 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.156777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.157052 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.157674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.159612 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.269883 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1332/2707] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:59 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:11:59.274796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.285411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458c992-d13d-4f34-bedf-cf2675a2a4fa admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.291612 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.292638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.292920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.293188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.293824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.295582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.305995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1376/2708] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:59.310201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.320587 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4873b0-633c-4204-8db9-8a7c51bbe591 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.325208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.325800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.326081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.326388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.327024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.328616 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.340584 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1333/2709] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:11:59.341318 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.343106 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:59.344322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.351591 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:59.359702 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53881e47-db3f-4d17-a6b4-688d1d6f529c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.361281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fff3c1-e738-4981-9ab6-4771b0de975e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.365628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362911c1-ad17-4dae-aa4d-0506317a1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.367669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362911c1-ad17-4dae-aa4d-0506317a1f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.367669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362911c1-ad17-4dae-aa4d-0506317a1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.367669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362911c1-ad17-4dae-aa4d-0506317a1f27 admin admin] PATH_INFO: `/v3/projects/14be7da72c8546858ecaee4b00ef70af/users/0f585cb741ae4a73919fb8f3d40fa52c/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.372942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.373580 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.373849 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.374109 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] PATH_INFO: `/v3/projects/80f7e6e5a0194c3aa602534a6c10be17` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.375453 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f7e6e5a0194c3aa602534a6c10be17)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.377784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.385156 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362911c1-ad17-4dae-aa4d-0506317a1f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14be7da72c8546858ecaee4b00ef70af, user_id=0f585cb741ae4a73919fb8f3d40fa52c, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.388147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362911c1-ad17-4dae-aa4d-0506317a1f27 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.480339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1334/2710] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:11:59 2022] PUT /identity/v3/projects/14be7da72c8546858ecaee4b00ef70af/users/0f585cb741ae4a73919fb8f3d40fa52c/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:59.490977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dc0e7-6f73-48ac-acb6-cbf60b22606a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.491260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dc0e7-6f73-48ac-acb6-cbf60b22606a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.491822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04dc0e7-6f73-48ac-acb6-cbf60b22606a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.493378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f7e6e5a0194c3aa602534a6c10be17', 'request_id': 'req-66c2cfb2-7d62-4e94-aa10-14f2073badce'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:59.499720 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f7e6e5a0194c3aa602534a6c10be17', 'request_id': 'req-66c2cfb2-7d62-4e94-aa10-14f2073badce'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:59.506575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f7e6e5a0194c3aa602534a6c10be17', 'request_id': 'req-66c2cfb2-7d62-4e94-aa10-14f2073badce'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:59.520500 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c04dc0e7-6f73-48ac-acb6-cbf60b22606a None None] MFA Rules not processed for user `0f585cb741ae4a73919fb8f3d40fa52c`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:11:59.521813 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c04dc0e7-6f73-48ac-acb6-cbf60b22606a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:59.584493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1335/2711] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:11:59.588539 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] The token cache is being invalidate because project 80f7e6e5a0194c3aa602534a6c10be17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:11:59.588834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:59.589510 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:11:59.589711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04dc0e7-6f73-48ac-acb6-cbf60b22606a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.591187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1377/2712] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:11:59 2022] DELETE /identity/v3/projects/80f7e6e5a0194c3aa602534a6c10be17 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:11:59.671210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04dc0e7-6f73-48ac-acb6-cbf60b22606a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.676140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.676757 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.677057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.677331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.677955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.680202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.765479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1336/2713] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:11:59 2022] POST /identity/v3/projects => generated 393 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:59.769301 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.770913 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:11:59.787638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2cfb2-7d62-4e94-aa10-14f2073badce admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.818970 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:11:59.858437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.860282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.860282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.860282 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.861283 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.864688 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.960888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1378/2714] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:11:59 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:11:59.964902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:11:59.976530 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8a1e9-3381-48ab-a88e-7ae4178e607c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:11:59.982118 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:11:59.983117 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:11:59.983397 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:11:59.983666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:11:59.984313 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:11:59.985887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:11:59.997024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1337/2715] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:12:00.001418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:00.011937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb483-6d52-4ce9-bed2-656cdec113f6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:00.016646 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:00.017323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:00.017603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:00.017877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] PATH_INFO: `/v3/projects/fd8ddb2cef654bf7b005ac5f34035ba7/users/b4fcaf85160744a3bbdba4d42ac20291/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:00.035074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8ddb2cef654bf7b005ac5f34035ba7, user_id=b4fcaf85160744a3bbdba4d42ac20291, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:00.037613 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:00.099967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1379/2716] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:12:00 2022] PUT /identity/v3/projects/fd8ddb2cef654bf7b005ac5f34035ba7/users/b4fcaf85160744a3bbdba4d42ac20291/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:00.104064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:00.113874 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e7109-da2b-4082-90f1-693f26f45fca admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:00.156631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:00.171754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5111d-a608-4750-91d3-664b013c1d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:00.172484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5111d-a608-4750-91d3-664b013c1d43 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:00.176786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5111d-a608-4750-91d3-664b013c1d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:00.176786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5111d-a608-4750-91d3-664b013c1d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:00.176786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5111d-a608-4750-91d3-664b013c1d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:00.176786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5111d-a608-4750-91d3-664b013c1d43 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:00.190838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1338/2717] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:12:00.192267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:00.204096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb3bc0-a615-466c-b081-fde9ac10dab9 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:00.209260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:00.211504 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:00.211865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:00.212220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] PATH_INFO: `/v3/projects/fd8ddb2cef654bf7b005ac5f34035ba7/users/b4fcaf85160744a3bbdba4d42ac20291/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:00.214426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8ddb2cef654bf7b005ac5f34035ba7, user_id=b4fcaf85160744a3bbdba4d42ac20291, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:00.217051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:00.240104 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1380/2718] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:12:00 2022] PUT /identity/v3/projects/fd8ddb2cef654bf7b005ac5f34035ba7/users/b4fcaf85160744a3bbdba4d42ac20291/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:12:00.240671 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:00.241263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:00.242129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:00.294908 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] MFA Rules not processed for user `b4fcaf85160744a3bbdba4d42ac20291`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:00.296080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:00.375673 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1339/2719] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:12:00.388693 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:00.391281 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:00.454337 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec5851-5b7a-4e57-8a15-72aca32e3ddd admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:00.459979 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad653179-067e-42d5-8b80-7238f4780dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:00.460711 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad653179-067e-42d5-8b80-7238f4780dee service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:00.460989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad653179-067e-42d5-8b80-7238f4780dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:00.461442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad653179-067e-42d5-8b80-7238f4780dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:00.470824 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad653179-067e-42d5-8b80-7238f4780dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:00.472236 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad653179-067e-42d5-8b80-7238f4780dee service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:00.482844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1381/2720] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:00 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:12:00.530711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:00.532177 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:00.544431 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e10312-896e-411e-b0f0-202a2ae4223a None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:00.554485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:00.554485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:00.554485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:00.554485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:00.555006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:00.610853 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:00.612303 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:00.622327 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1340/2721] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:12:01.424163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dc214-3e8c-42a7-ab82-d7c4851ca0ba None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:01.424638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dc214-3e8c-42a7-ab82-d7c4851ca0ba None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:01.424736 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2dc214-3e8c-42a7-ab82-d7c4851ca0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:01.444260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-bc2dc214-3e8c-42a7-ab82-d7c4851ca0ba None None] MFA Rules not processed for user `9ca8c5d1971a4ec1a448f2081ceb835b`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:01.446245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bc2dc214-3e8c-42a7-ab82-d7c4851ca0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:01.508285 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1382/2722] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:05.500294 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:05.576898 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902e08-38bf-4084-ba36-2d14106dd430 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:05.582330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:05.583041 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:05.583433 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:05.584429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] PATH_INFO: `/v3/users/0f585cb741ae4a73919fb8f3d40fa52c` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:05.586605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f585cb741ae4a73919fb8f3d40fa52c)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:05.588801 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:05.595696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=100 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:05.878633 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f585cb741ae4a73919fb8f3d40fa52c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:05.884187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f585cb741ae4a73919fb8f3d40fa52c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:05.888162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f585cb741ae4a73919fb8f3d40fa52c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:05.978143 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f585cb741ae4a73919fb8f3d40fa52c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:05.987599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f585cb741ae4a73919fb8f3d40fa52c'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:06.011447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1341/2723] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:12:05 2022] DELETE /identity/v3/users/0f585cb741ae4a73919fb8f3d40fa52c => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:06.015981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:06.016482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:06.016752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:06.049351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:06.050825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:06.076922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:06.124937 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1383/2724] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:12:06.134786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:06.136544 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:06.224376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8f151-c253-4aa5-af92-d0fef90c56ac admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:06.228861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:06.229496 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:06.229775 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:06.230049 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:06.242210 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:06.243845 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:06.254751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1342/2725] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:12:06.808098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:06.810329 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:06.822814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbe149-d04d-44b5-a014-698559f88ef1 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:06.827628 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:06.828450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:06.828728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:06.828999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] PATH_INFO: `/v3/projects/14be7da72c8546858ecaee4b00ef70af` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:06.830208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14be7da72c8546858ecaee4b00ef70af)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:06.832138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:06.994833 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14be7da72c8546858ecaee4b00ef70af', 'request_id': 'req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.001400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14be7da72c8546858ecaee4b00ef70af', 'request_id': 'req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.008110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14be7da72c8546858ecaee4b00ef70af', 'request_id': 'req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.082496 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] The token cache is being invalidate because project 14be7da72c8546858ecaee4b00ef70af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:12:07.085865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.085865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.085865 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1384/2726] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:12:06 2022] DELETE /identity/v3/projects/14be7da72c8546858ecaee4b00ef70af => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:07.089655 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:07.099518 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fbb07-f318-4263-85fd-3ffda8959fbd service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:07.165207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:07.166116 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:07.166409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:07.166685 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] PATH_INFO: `/v3/users/b4fcaf85160744a3bbdba4d42ac20291` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:07.168045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fcaf85160744a3bbdba4d42ac20291)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:07.169955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:07.289863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:07.394936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fcaf85160744a3bbdba4d42ac20291'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.395735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fcaf85160744a3bbdba4d42ac20291'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.396371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fcaf85160744a3bbdba4d42ac20291'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.486667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fcaf85160744a3bbdba4d42ac20291'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.495504 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f60c119a-28f7-4778-9001-f710bdeca6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fcaf85160744a3bbdba4d42ac20291'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:07.513721 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1343/2727] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:12:07 2022] DELETE /identity/v3/users/b4fcaf85160744a3bbdba4d42ac20291 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 22 11:12:08.123918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:08.125006 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:08.141950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a780d-fc7c-4463-83e9-bb8ecad542a1 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:08.202526 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:08.203156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:08.203434 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:08.203739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] PATH_INFO: `/v3/projects/fd8ddb2cef654bf7b005ac5f34035ba7` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:08.204845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8ddb2cef654bf7b005ac5f34035ba7)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:08.206831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:08.357130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8ddb2cef654bf7b005ac5f34035ba7', 'request_id': 'req-e4565908-53f7-4bc1-86ae-4b4aed363fe8'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:08.359138 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:08.362905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8ddb2cef654bf7b005ac5f34035ba7', 'request_id': 'req-e4565908-53f7-4bc1-86ae-4b4aed363fe8'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:08.370691 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8ddb2cef654bf7b005ac5f34035ba7', 'request_id': 'req-e4565908-53f7-4bc1-86ae-4b4aed363fe8'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:08.464817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] The token cache is being invalidate because project fd8ddb2cef654bf7b005ac5f34035ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:12:08.465112 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:08.468934 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-e4565908-53f7-4bc1-86ae-4b4aed363fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:08.470629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1385/2728] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:12:08 2022] DELETE /identity/v3/projects/fd8ddb2cef654bf7b005ac5f34035ba7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:08.475572 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb8e7e-c5d4-4dd0-8465-98ee44a0850f None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:08.479323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb8e7e-c5d4-4dd0-8465-98ee44a0850f None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:08.479323 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb8e7e-c5d4-4dd0-8465-98ee44a0850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:08.509551 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-aafb8e7e-c5d4-4dd0-8465-98ee44a0850f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:08.510917 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-aafb8e7e-c5d4-4dd0-8465-98ee44a0850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:08.574171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1344/2729] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:12:08.582117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6d8e3-3328-4456-b002-25281a02fc89 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:08.582469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6d8e3-3328-4456-b002-25281a02fc89 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:08.582743 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6d8e3-3328-4456-b002-25281a02fc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:08.603438 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-b2e6d8e3-3328-4456-b002-25281a02fc89 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:08.604647 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b2e6d8e3-3328-4456-b002-25281a02fc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:08.612465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1386/2730] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:08.617626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb8e7e-c5d4-4dd0-8465-98ee44a0850f None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:08.627300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb8e7e-c5d4-4dd0-8465-98ee44a0850f None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:08.631787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:08.632654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:08.632946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:08.633219 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:08.633862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:12:08.634164 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:08.635788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:08.647628 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1345/2731] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:12:08.652089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6d8e3-3328-4456-b002-25281a02fc89 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:08.661139 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6d8e3-3328-4456-b002-25281a02fc89 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:08.666463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:08.667057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:08.667331 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:08.667598 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:08.668209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:08.670077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:08.780472 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1387/2732] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:12:08 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:12:08.786588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:08.797787 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e713e0-828d-45b7-bd9c-a5f125a3d10c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:08.855026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:08.855636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:08.855911 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:08.856215 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:08.856833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:08.858804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:08.959724 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1346/2733] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:12:08 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:12:08.964577 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:08.974766 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a93e-4a7f-4720-bdb3-72eaa04f90e7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:08.980031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:08.980621 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:08.980896 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:08.981166 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:08.981796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:08.983483 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:08.993641 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1388/2734] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:12:08.997621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:09.007702 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b974e8a-92ea-46b4-bc4f-4872db08a74c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:09.012495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b47ef6-e849-4be6-a1a0-74a87f712a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:09.013081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47ef6-e849-4be6-a1a0-74a87f712a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:09.013358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47ef6-e849-4be6-a1a0-74a87f712a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:09.013627 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b47ef6-e849-4be6-a1a0-74a87f712a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:09.014315 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b47ef6-e849-4be6-a1a0-74a87f712a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:09.016087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b47ef6-e849-4be6-a1a0-74a87f712a12 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:09.027438 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1347/2735] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:12:09.031523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:09.042219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2dc1e4-053d-4764-baea-0aa7e80a7f86 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:09.046774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:09.047411 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:09.047687 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:09.048081 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] PATH_INFO: `/v3/projects/3416bfb874054b23b599b030e7c223df/users/c639668698db455e95b0072030aada98/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:09.062694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3416bfb874054b23b599b030e7c223df, user_id=c639668698db455e95b0072030aada98, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:09.065317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:09.131565 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1389/2736] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:12:09 2022] PUT /identity/v3/projects/3416bfb874054b23b599b030e7c223df/users/c639668698db455e95b0072030aada98/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:09.197986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800a941-1e7b-4e3d-8a78-576c85c84bf9 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:09.198386 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800a941-1e7b-4e3d-8a78-576c85c84bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:09.198664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800a941-1e7b-4e3d-8a78-576c85c84bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:09.219594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:09.227674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4800a941-1e7b-4e3d-8a78-576c85c84bf9 None None] MFA Rules not processed for user `c639668698db455e95b0072030aada98`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:09.229058 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4800a941-1e7b-4e3d-8a78-576c85c84bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:09.293681 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1348/2737] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:09.304768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:09.306555 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:09.363036 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471dc27-3c76-477c-a979-ad806b025b69 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:09.366741 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd53f74-aaf2-43ca-befe-689ff308a204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:09.367604 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53f74-aaf2-43ca-befe-689ff308a204 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:09.367880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53f74-aaf2-43ca-befe-689ff308a204 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:09.368150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd53f74-aaf2-43ca-befe-689ff308a204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:09.376273 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd53f74-aaf2-43ca-befe-689ff308a204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:09.377684 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd53f74-aaf2-43ca-befe-689ff308a204 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:09.386869 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1390/2738] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:12:19.217449 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800a941-1e7b-4e3d-8a78-576c85c84bf9 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:19.339731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800a941-1e7b-4e3d-8a78-576c85c84bf9 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:19.344388 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:19.345024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:19.345298 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:19.345566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] PATH_INFO: `/v3/users/c639668698db455e95b0072030aada98` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:19.346712 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c639668698db455e95b0072030aada98)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:19.348610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:19.382889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=59,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:19.608900 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c639668698db455e95b0072030aada98'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:19.609480 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c639668698db455e95b0072030aada98'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:19.610081 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c639668698db455e95b0072030aada98'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:19.663313 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c639668698db455e95b0072030aada98'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:19.669228 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:19.671417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c639668698db455e95b0072030aada98'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:19.688597 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1349/2739] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:12:19 2022] DELETE /identity/v3/users/c639668698db455e95b0072030aada98 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:19.694131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:19.694131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:19.694131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:19.746200 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:19.747493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:19.808014 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1391/2740] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:19.817751 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:19.819237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:19.894594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324692d-6ce1-469e-ad89-d6e72ea4cae0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:19.898201 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6358f-5baa-4ced-beda-f757d0ce5042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:19.898854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6358f-5baa-4ced-beda-f757d0ce5042 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:19.899137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6358f-5baa-4ced-beda-f757d0ce5042 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:19.899409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6358f-5baa-4ced-beda-f757d0ce5042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:19.912931 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6358f-5baa-4ced-beda-f757d0ce5042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:19.916038 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6358f-5baa-4ced-beda-f757d0ce5042 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:19.924428 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1350/2741] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:12:20.361506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:20.363505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:20.380951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf595ed-6ecb-484e-9a4a-aa8e1c75f264 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:20.382774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:20.383415 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:20.383995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:20.383995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] PATH_INFO: `/v3/projects/3416bfb874054b23b599b030e7c223df` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:20.385165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3416bfb874054b23b599b030e7c223df)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:20.387026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:20.504968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3416bfb874054b23b599b030e7c223df', 'request_id': 'req-4d9130d7-6246-4393-bd62-e2ed36cb08ab'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:20.510594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3416bfb874054b23b599b030e7c223df', 'request_id': 'req-4d9130d7-6246-4393-bd62-e2ed36cb08ab'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:20.517002 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3416bfb874054b23b599b030e7c223df', 'request_id': 'req-4d9130d7-6246-4393-bd62-e2ed36cb08ab'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:20.587538 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] The token cache is being invalidate because project 3416bfb874054b23b599b030e7c223df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:12:20.587839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:20.588465 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4d9130d7-6246-4393-bd62-e2ed36cb08ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:20.589946 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1392/2742] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:12:20 2022] DELETE /identity/v3/projects/3416bfb874054b23b599b030e7c223df => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:20.596142 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751db3-a445-4a3b-bab4-a3d526a2fa56 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:20.596656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751db3-a445-4a3b-bab4-a3d526a2fa56 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:20.596794 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751db3-a445-4a3b-bab4-a3d526a2fa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:20.644748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d3751db3-a445-4a3b-bab4-a3d526a2fa56 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:20.645885 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d3751db3-a445-4a3b-bab4-a3d526a2fa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:20.702373 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1351/2743] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:20.709967 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbdcea-c5ce-4c9d-80fe-6133daa4af86 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:20.710271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbdcea-c5ce-4c9d-80fe-6133daa4af86 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:20.710552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbdcea-c5ce-4c9d-80fe-6133daa4af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:20.731219 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-debbdcea-c5ce-4c9d-80fe-6133daa4af86 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:20.732424 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-debbdcea-c5ce-4c9d-80fe-6133daa4af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:20.740267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1393/2744] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:20.744854 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751db3-a445-4a3b-bab4-a3d526a2fa56 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:20.755307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751db3-a445-4a3b-bab4-a3d526a2fa56 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:20.760105 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:20.765546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:20.765546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:20.765546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:20.765546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:12:20.765546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:20.765546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:20.776585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1352/2745] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:12:20.780899 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbdcea-c5ce-4c9d-80fe-6133daa4af86 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:20.790073 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbdcea-c5ce-4c9d-80fe-6133daa4af86 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:20.796413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:20.797010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:20.797292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:20.797567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:20.798198 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:20.800142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:20.899203 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1394/2746] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:12:20 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:12:20.904140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:20.914103 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807885e-a61a-43a7-9fe9-d6c6101036e6 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:20.940062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:20.971996 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:20.972599 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:20.972879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:20.973146 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:20.973754 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:20.975657 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:21.075536 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1353/2747] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:12:20 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:12:21.079710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:21.090796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68928a0-f17b-43c1-b9b9-1c5813bd7b76 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:21.095409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:21.096007 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:21.096544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:21.096544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:21.097181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:21.098799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:21.109251 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1395/2748] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:12:21.113403 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:21.124370 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916766b5-1097-44dd-86f0-1eddc8e003a0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:21.129088 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7751c3-5858-47b7-a136-7c35ee23686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:21.129691 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7751c3-5858-47b7-a136-7c35ee23686e admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:21.129981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7751c3-5858-47b7-a136-7c35ee23686e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:21.130265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7751c3-5858-47b7-a136-7c35ee23686e admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:21.130933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7751c3-5858-47b7-a136-7c35ee23686e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:21.136674 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7751c3-5858-47b7-a136-7c35ee23686e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:21.143590 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1354/2749] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:12:21.147812 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:21.168209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39df193-d3bd-453d-8777-6a0e29c39208 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:21.168209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:21.168209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:21.168209 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:21.168673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] PATH_INFO: `/v3/projects/c770e206ef4142ca97e2ef050d5cbca1/users/09694c2ab49f49749b47657365df0b56/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:21.179735 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c770e206ef4142ca97e2ef050d5cbca1, user_id=09694c2ab49f49749b47657365df0b56, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:21.182480 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:21.184774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:21.303623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1396/2750] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:12:21 2022] PUT /identity/v3/projects/c770e206ef4142ca97e2ef050d5cbca1/users/09694c2ab49f49749b47657365df0b56/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:21.313696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:21.313980 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:21.314275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:21.343057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] MFA Rules not processed for user `09694c2ab49f49749b47657365df0b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:21.344270 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:21.410322 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1355/2751] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:21.425724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:21.425724 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:21.492150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ee74d-c1ff-4524-8263-dad590ceff46 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:21.496358 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990660ce5a9f4213a5b42b3824d0b567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:21.497091 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:21.497375 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:21.497645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:21.506147 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:21.507624 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:21.516728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1397/2752] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:12:22.923306 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:22.924726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:22.982039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc1e6b-37e9-4b26-9e2f-e45a87f60ad9 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:22.985581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:22.986268 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:22.986558 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:22.986826 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:22.988718 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:23.038376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:23.052334 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:23.053756 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:23.064017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1356/2753] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:12:28.096715 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:28.098170 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:28.192018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80350ab1-df26-4b5f-9e06-bb4faa85e46b service glance] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:28.197137 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe0b1f-e4ac-4071-a2a4-1f61d7784f5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:28.197739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0b1f-e4ac-4071-a2a4-1f61d7784f5a service placement] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:28.198021 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0b1f-e4ac-4071-a2a4-1f61d7784f5a service placement] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:28.198330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0b1f-e4ac-4071-a2a4-1f61d7784f5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:28.200939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7afe0b1f-e4ac-4071-a2a4-1f61d7784f5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:28.212778 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe0b1f-e4ac-4071-a2a4-1f61d7784f5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:28.214392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe0b1f-e4ac-4071-a2a4-1f61d7784f5a service placement] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:28.224912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1398/2754] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:12:33.148132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=6 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:38.220585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:INSERT=1,keystone:SELECT=69 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:12:54.369297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:54.435205 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081bbb6-52a0-4335-997d-84ae68ab3a39 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:54.439839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:54.440473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:54.440748 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:54.441013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] PATH_INFO: `/v3/users/09694c2ab49f49749b47657365df0b56` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:54.443129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09694c2ab49f49749b47657365df0b56)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:54.445064 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:54.596209 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09694c2ab49f49749b47657365df0b56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:54.596777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09694c2ab49f49749b47657365df0b56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:54.597376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09694c2ab49f49749b47657365df0b56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:54.680185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09694c2ab49f49749b47657365df0b56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:54.687709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09694c2ab49f49749b47657365df0b56'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:54.705123 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1357/2755] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:12:54 2022] DELETE /identity/v3/users/09694c2ab49f49749b47657365df0b56 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:12:54.709703 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:54.710182 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:54.710309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:54.729908 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:12:54.731131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:54.794132 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1399/2756] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:12:54.804575 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:54.805990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:54.885171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802834d-24a5-41b4-a121-97fdfddf537f admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:54.888951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:54.889565 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:54.889844 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:54.890113 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:54.899473 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:54.900894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:54.910151 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1358/2757] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:12:55.345556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:12:55.347076 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:12:55.361245 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d0340-7f59-486a-96f4-150dc94740b4 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:12:55.366072 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:12:55.367638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:12:55.372392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:12:55.372392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] PATH_INFO: `/v3/projects/c770e206ef4142ca97e2ef050d5cbca1` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:12:55.372392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c770e206ef4142ca97e2ef050d5cbca1)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:12:55.372392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:12:55.498211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c770e206ef4142ca97e2ef050d5cbca1', 'request_id': 'req-b4d36902-6e60-461a-870c-7949ef198981'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:55.503753 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c770e206ef4142ca97e2ef050d5cbca1', 'request_id': 'req-b4d36902-6e60-461a-870c-7949ef198981'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:55.510436 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c770e206ef4142ca97e2ef050d5cbca1', 'request_id': 'req-b4d36902-6e60-461a-870c-7949ef198981'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:55.605408 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] The token cache is being invalidate because project c770e206ef4142ca97e2ef050d5cbca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:12:55.605757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:55.606488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:12:55.607995 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1400/2758] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:12:55 2022] DELETE /identity/v3/projects/c770e206ef4142ca97e2ef050d5cbca1 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:13:04.906043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:13:05.602000 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:14:30.457987 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:14:30.466879 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f344ad-ad58-4ced-80ad-d56ae4c344ce service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:14:30.516577 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21efc7-b2eb-44a7-b492-38bfcf90339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:14:30.517175 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21efc7-b2eb-44a7-b492-38bfcf90339c service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:14:30.517456 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21efc7-b2eb-44a7-b492-38bfcf90339c service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:14:30.517726 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21efc7-b2eb-44a7-b492-38bfcf90339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:14:30.519562 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5a21efc7-b2eb-44a7-b492-38bfcf90339c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:30.607782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21efc7-b2eb-44a7-b492-38bfcf90339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:14:30.609807 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21efc7-b2eb-44a7-b492-38bfcf90339c service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:14:30.630727 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1359/2759] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:14:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:14:31.808172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:14:31.809655 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:31.823026 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d36902-6e60-461a-870c-7949ef198981 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:14:31.826972 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77470d-fb38-414c-a1c2-934f6ef60c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:14:31.827574 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77470d-fb38-414c-a1c2-934f6ef60c1a service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:14:31.828457 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77470d-fb38-414c-a1c2-934f6ef60c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:14:31.828804 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77470d-fb38-414c-a1c2-934f6ef60c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:14:31.832531 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-bf77470d-fb38-414c-a1c2-934f6ef60c1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:31.846312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77470d-fb38-414c-a1c2-934f6ef60c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:14:31.847730 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77470d-fb38-414c-a1c2-934f6ef60c1a service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:14:31.859807 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1401/2760] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:14:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:14:35.285317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9dc11-15cf-4873-b5a9-e650ac5c4fc2 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:14:35.285796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9dc11-15cf-4873-b5a9-e650ac5c4fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:14:35.285901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9dc11-15cf-4873-b5a9-e650ac5c4fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:14:35.304934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-84f9dc11-15cf-4873-b5a9-e650ac5c4fc2 None None] MFA Rules not processed for user `9ca8c5d1971a4ec1a448f2081ceb835b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:14:35.314065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-84f9dc11-15cf-4873-b5a9-e650ac5c4fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:35.360477 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1360/2761] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:14:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:14:41.853844 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=6 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:14:45.353499 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:14:55.769241 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:14:55.769707 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:14:55.769821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:14:55.796525 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] MFA Rules not processed for user `2f8117e04a1d4f7aa802df3dda87606e`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:14:55.798662 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:55.823471 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9dc11-15cf-4873-b5a9-e650ac5c4fc2 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:14:55.836833 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9dc11-15cf-4873-b5a9-e650ac5c4fc2 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:14:55.840920 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:14:55.841653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:14:55.842559 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:14:55.843686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:14:55.846013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:55.858460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1402/2762] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:14:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:14:55.859830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:14:55.861132 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:14:55.870846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1361/2763] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:14:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:14:56.204803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:14:56.206344 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:56.220497 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a7a7f-efd1-4968-84b3-7561162ecd65 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:14:56.224977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:14:56.225593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:14:56.225871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:14:56.226143 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:14:56.229167 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:56.240923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:14:56.241798 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:14:56.251799 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1403/2764] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:14:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:14:56.731382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:14:56.732878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:56.806834 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d1332-6b2a-44ef-9ac4-00607f6ad6c8 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:14:56.812512 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:14:56.813185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:14:56.813535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:14:56.813872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:14:56.815680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:14:56.827847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:14:56.829421 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:14:56.839275 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1362/2765] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:14:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:15:00.716925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:15:00.718987 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:15:00.735947 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aeb05-c4c1-4d7e-8244-69f08393a3ce service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:15:00.741894 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:15:00.742619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:15:00.743051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:15:00.743319 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:15:00.745240 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:15:00.758224 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:15:00.761514 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:15:00.771562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1404/2766] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:15:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:15:06.208636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:15:06.221092 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0176a9c-c7e8-4ff9-9582-de584b886fa5 service placement] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:15:06.226550 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:15:06.227171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:15:06.227434 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:15:06.227709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:15:06.229406 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:15:06.241789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:15:06.243384 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:15:06.254659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1363/2767] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:15:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:15:10.766857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=37 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:15:16.249696 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=44 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:23.360160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:23.369874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:23.430276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc54ab-012b-4e57-9d4a-96dc7a9f7c6e service nova] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:23.434960 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:23.435579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:23.435857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:23.436126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] PATH_INFO: `/v3/users/2f8117e04a1d4f7aa802df3dda87606e` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:23.437221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8117e04a1d4f7aa802df3dda87606e)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:23.439158 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:23.625312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8117e04a1d4f7aa802df3dda87606e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:23.625917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8117e04a1d4f7aa802df3dda87606e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:23.626549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8117e04a1d4f7aa802df3dda87606e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:23.633800 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8117e04a1d4f7aa802df3dda87606e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:23.705731 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-be539085-900f-4a36-803e-e5a715f2dbfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8117e04a1d4f7aa802df3dda87606e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:23.724556 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1405/2768] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:16:23 2022] DELETE /identity/v3/users/2f8117e04a1d4f7aa802df3dda87606e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:16:23.955822 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:23.970828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e6bf5-622c-4d9a-82b8-9fb0541b88bf service nova] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:24.028453 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:24.029087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.029367 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.029641 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] PATH_INFO: `/v3/projects/6a293d557d674e8d97596857b8d179b3` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.030776 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a293d557d674e8d97596857b8d179b3)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:24.032687 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:24.153875 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a293d557d674e8d97596857b8d179b3', 'request_id': 'req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:24.159566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a293d557d674e8d97596857b8d179b3', 'request_id': 'req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:24.165835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a293d557d674e8d97596857b8d179b3', 'request_id': 'req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:24.256872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] The token cache is being invalidate because project 6a293d557d674e8d97596857b8d179b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:16:24.257167 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:24.257789 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-d23a2c84-38bb-4b04-bc9a-9bbafbb26c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:16:24.259335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1364/2769] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:16:23 2022] DELETE /identity/v3/projects/6a293d557d674e8d97596857b8d179b3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:16:24.264211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9333-be9c-41ce-aa1d-ca8be4551fc2 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.264528 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9333-be9c-41ce-aa1d-ca8be4551fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.264805 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb9333-be9c-41ce-aa1d-ca8be4551fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.284523 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e8bb9333-be9c-41ce-aa1d-ca8be4551fc2 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:16:24.285875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e8bb9333-be9c-41ce-aa1d-ca8be4551fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:24.344050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1406/2770] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:16:24.351878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd506b9d-d3b5-454a-a44d-d2470bd38692 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.352162 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd506b9d-d3b5-454a-a44d-d2470bd38692 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.352436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd506b9d-d3b5-454a-a44d-d2470bd38692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.372208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-bd506b9d-d3b5-454a-a44d-d2470bd38692 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:16:24.373402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-bd506b9d-d3b5-454a-a44d-d2470bd38692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:24.381981 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1365/2771] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:16:24.386428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9333-be9c-41ce-aa1d-ca8be4551fc2 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:24.396737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb9333-be9c-41ce-aa1d-ca8be4551fc2 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:24.401336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:24.403748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.403748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.403748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.403748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:16:24.403748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:24.405405 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:24.417477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1407/2772] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:16:24.421869 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd506b9d-d3b5-454a-a44d-d2470bd38692 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:24.432016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd506b9d-d3b5-454a-a44d-d2470bd38692 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:24.436634 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:24.437376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.437658 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.437933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.438605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:24.440509 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:24.532035 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1366/2773] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:16:24 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:16:24.536998 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:24.547220 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed65607-4179-4221-b599-8b5d54bbf026 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:24.550875 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:24.603587 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:24.604202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.604475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.604747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.605363 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:24.607299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:24.708631 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1408/2774] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:16:24 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:16:24.712963 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:24.723445 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ac45-abd4-422b-9eb3-9abb63fb99dd admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:24.729171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:24.729767 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.730043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.730341 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.730979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:24.732546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:24.742452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1367/2775] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:16:24.746477 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:24.757157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed3f54-6ca0-499a-830a-2a541c2dfd9c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:24.761763 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a0af0-bffb-43ec-a453-42f88f2614e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:24.762386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a0af0-bffb-43ec-a453-42f88f2614e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.762680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a0af0-bffb-43ec-a453-42f88f2614e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.762956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a0af0-bffb-43ec-a453-42f88f2614e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.763640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a0af0-bffb-43ec-a453-42f88f2614e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:24.765293 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a0af0-bffb-43ec-a453-42f88f2614e3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:24.777326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1409/2776] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:16:24.781407 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:24.791613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941065e7-4844-40c6-b411-942c48febb99 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:24.796300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:24.797009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.797374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.797646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] PATH_INFO: `/v3/projects/0ae1678cd50c4c019bd146af352c55f1/users/87af852c54744ecf928c56b5b5a3daa5/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.812380 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae1678cd50c4c019bd146af352c55f1, user_id=87af852c54744ecf928c56b5b5a3daa5, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:24.815044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:24.931953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1368/2777] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:16:24 2022] PUT /identity/v3/projects/0ae1678cd50c4c019bd146af352c55f1/users/87af852c54744ecf928c56b5b5a3daa5/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:16:24.943852 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c971de-378d-4527-801c-a7ee7a680630 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:24.944140 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c971de-378d-4527-801c-a7ee7a680630 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:24.944420 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c971de-378d-4527-801c-a7ee7a680630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:24.972266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-21c971de-378d-4527-801c-a7ee7a680630 None None] MFA Rules not processed for user `87af852c54744ecf928c56b5b5a3daa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:16:24.974202 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-21c971de-378d-4527-801c-a7ee7a680630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:25.036912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1410/2778] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:16:25.041415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:25.060009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:25.101046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af07ef2-2d0b-47be-8ca7-540cd2f23250 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:25.105563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:25.106163 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.106465 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.106735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.107459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:25.109329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:25.234001 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1369/2779] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:16:25 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:16:25.238286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c971de-378d-4527-801c-a7ee7a680630 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:25.250698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c971de-378d-4527-801c-a7ee7a680630 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:25.306204 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:25.306832 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.307110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.307391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.308008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:25.309888 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:25.416681 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1411/2780] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:16:25 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:16:25.420585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:25.431739 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38542940-baf5-4462-8fe4-24e2b5a7e8f2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:25.436488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:25.437080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.437358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.437659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.438333 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:25.439921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:25.450368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1370/2781] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:16:25.454479 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:25.460876 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:25.464659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9ca33-9541-4861-beaa-1355b2dff647 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:25.469253 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:25.474321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.474321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.474321 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] PATH_INFO: `/v3/projects/17bb4fe7a63445e1a44beb5d0fba6b3a/users/4aee5b2d8b69495e87adc03b268d3fe2/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.485928 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bb4fe7a63445e1a44beb5d0fba6b3a, user_id=4aee5b2d8b69495e87adc03b268d3fe2, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:25.489213 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:25.559683 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1412/2782] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:16:25 2022] PUT /identity/v3/projects/17bb4fe7a63445e1a44beb5d0fba6b3a/users/4aee5b2d8b69495e87adc03b268d3fe2/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:16:25.564349 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:25.579493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c2724-1d29-4b11-aead-9c39fdc14418 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:25.638720 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcdf09-6bce-4677-af09-c28d3b923735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:25.639325 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcdf09-6bce-4677-af09-c28d3b923735 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.639605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcdf09-6bce-4677-af09-c28d3b923735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.639878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcdf09-6bce-4677-af09-c28d3b923735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.640508 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcdf09-6bce-4677-af09-c28d3b923735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:25.642101 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcdf09-6bce-4677-af09-c28d3b923735 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:25.651570 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1371/2783] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:16:25.655831 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:25.667010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270ebb-1626-4e77-8145-c9cce2c6707f admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:25.671593 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:25.672239 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.672520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.672796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] PATH_INFO: `/v3/projects/17bb4fe7a63445e1a44beb5d0fba6b3a/users/4aee5b2d8b69495e87adc03b268d3fe2/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.674769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bb4fe7a63445e1a44beb5d0fba6b3a, user_id=4aee5b2d8b69495e87adc03b268d3fe2, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:25.678188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:25.688609 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1413/2784] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:16:25 2022] PUT /identity/v3/projects/17bb4fe7a63445e1a44beb5d0fba6b3a/users/4aee5b2d8b69495e87adc03b268d3fe2/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:16:25.699082 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cb8dd-a246-4af1-9982-478e8c13e94e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.699371 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cb8dd-a246-4af1-9982-478e8c13e94e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.699650 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cb8dd-a246-4af1-9982-478e8c13e94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.727621 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f93cb8dd-a246-4af1-9982-478e8c13e94e None None] MFA Rules not processed for user `4aee5b2d8b69495e87adc03b268d3fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:16:25.728828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f93cb8dd-a246-4af1-9982-478e8c13e94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:25.794760 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1372/2785] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:16:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:16:25.806010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:25.807782 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:25.886107 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a07ce-a109-4324-a7ad-a9eefe32d52e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:25.889924 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:25.890560 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:25.890836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:25.891101 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:25.900728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:25.902145 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:25.911905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1414/2786] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:16:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:16:26.114015 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cb8dd-a246-4af1-9982-478e8c13e94e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:26.124237 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cb8dd-a246-4af1-9982-478e8c13e94e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:26.127924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3aaee-7782-41d2-adad-0a95024b7c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:26.128524 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3aaee-7782-41d2-adad-0a95024b7c64 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:26.128805 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3aaee-7782-41d2-adad-0a95024b7c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:26.129592 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3aaee-7782-41d2-adad-0a95024b7c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:26.139187 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:26.185605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3aaee-7782-41d2-adad-0a95024b7c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:26.187027 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3aaee-7782-41d2-adad-0a95024b7c64 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:26.196852 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1373/2787] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:16:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:16:35.907252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:36.219954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=22 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:39.490821 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:16:39.492803 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:39.504049 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9558c2-80a0-4d9c-a982-769f7c82654f service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:16:39.509013 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:16:39.509788 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:39.509953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:39.510252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:39.511955 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:39.578469 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:16:39.580030 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:16:39.589074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1415/2788] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:16:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:16:43.522281 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793f9e-7cad-4a8e-aa38-47cbc166d8d0 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:16:43.522872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793f9e-7cad-4a8e-aa38-47cbc166d8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:16:43.522872 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793f9e-7cad-4a8e-aa38-47cbc166d8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:16:43.542057 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-c5793f9e-7cad-4a8e-aa38-47cbc166d8d0 None None] MFA Rules not processed for user `9ca8c5d1971a4ec1a448f2081ceb835b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:16:43.543305 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-c5793f9e-7cad-4a8e-aa38-47cbc166d8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:16:43.550159 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1374/2789] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:16:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:16:49.584887 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=28 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:16:53.536166 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:17.773336 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:17.835674 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfaa70-db6f-466a-a823-b8e36deb7502 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:17.840391 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:17.841012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:17.841292 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:17.841567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] PATH_INFO: `/v3/users/87af852c54744ecf928c56b5b5a3daa5` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:17.842787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87af852c54744ecf928c56b5b5a3daa5)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:17.844679 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:18.137699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87af852c54744ecf928c56b5b5a3daa5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:18.138392 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87af852c54744ecf928c56b5b5a3daa5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:18.138993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87af852c54744ecf928c56b5b5a3daa5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:18.145757 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87af852c54744ecf928c56b5b5a3daa5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:18.233467 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87af852c54744ecf928c56b5b5a3daa5'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:18.252086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1416/2790] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:17:17 2022] DELETE /identity/v3/users/87af852c54744ecf928c56b5b5a3daa5 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:18.256464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:18.256959 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:18.257068 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:18.276045 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:18.277217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:18.334520 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1375/2791] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:17:18.344756 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:18.346278 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:18.407317 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b8c81-02de-4151-ae58-4beb41711b66 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:18.410916 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:18.411518 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:18.411796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:18.412069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:18.421993 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:18.423535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:18.432839 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1417/2792] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:17:18.837447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:18.838975 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:18.851031 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748e6e-9a0f-4890-becd-b471a9ccea9c None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:18.860083 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:18.860689 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:18.861110 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:18.861271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] PATH_INFO: `/v3/projects/0ae1678cd50c4c019bd146af352c55f1` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:18.862485 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae1678cd50c4c019bd146af352c55f1)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:18.864357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:18.956670 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae1678cd50c4c019bd146af352c55f1', 'request_id': 'req-3164d64a-2946-4f3d-8c17-2d79851e38e2'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:18.962626 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae1678cd50c4c019bd146af352c55f1', 'request_id': 'req-3164d64a-2946-4f3d-8c17-2d79851e38e2'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:18.968812 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae1678cd50c4c019bd146af352c55f1', 'request_id': 'req-3164d64a-2946-4f3d-8c17-2d79851e38e2'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.040729 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] The token cache is being invalidate because project 0ae1678cd50c4c019bd146af352c55f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:17:19.041011 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.041624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.043125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1376/2793] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:17:18 2022] DELETE /identity/v3/projects/0ae1678cd50c4c019bd146af352c55f1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:19.047118 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:19.057165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edc8bc-62f5-4b14-b1a4-48ec8105d96d service neutron] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:19.075079 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:19.110920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:19.111535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:19.111796 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:19.112067 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] PATH_INFO: `/v3/users/4aee5b2d8b69495e87adc03b268d3fe2` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:19.113163 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aee5b2d8b69495e87adc03b268d3fe2)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:19.115088 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:19.297828 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aee5b2d8b69495e87adc03b268d3fe2'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.298487 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aee5b2d8b69495e87adc03b268d3fe2'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.299087 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aee5b2d8b69495e87adc03b268d3fe2'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.306575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aee5b2d8b69495e87adc03b268d3fe2'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.387290 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-1c57254c-3274-4e51-bbb4-2ad1d65494a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aee5b2d8b69495e87adc03b268d3fe2'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:19.405596 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1418/2794] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:17:19 2022] DELETE /identity/v3/users/4aee5b2d8b69495e87adc03b268d3fe2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:19.805529 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:19.807090 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:19.820086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3164d64a-2946-4f3d-8c17-2d79851e38e2 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:19.878490 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:19.879134 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:19.879419 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:19.879690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] PATH_INFO: `/v3/projects/17bb4fe7a63445e1a44beb5d0fba6b3a` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:19.880830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bb4fe7a63445e1a44beb5d0fba6b3a)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:19.882765 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:19.980454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:19.996796 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bb4fe7a63445e1a44beb5d0fba6b3a', 'request_id': 'req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:20.002541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bb4fe7a63445e1a44beb5d0fba6b3a', 'request_id': 'req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:20.009096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bb4fe7a63445e1a44beb5d0fba6b3a', 'request_id': 'req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:20.063614 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] The token cache is being invalidate because project 17bb4fe7a63445e1a44beb5d0fba6b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:17:20.063901 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:20.064539 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9d67d3a1-a032-4b67-83c2-7a8f83bd6efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:20.066010 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1377/2795] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:17:19 2022] DELETE /identity/v3/projects/17bb4fe7a63445e1a44beb5d0fba6b3a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:20.071680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871b117-c94a-4666-9764-68f4387b3d60 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.072150 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871b117-c94a-4666-9764-68f4387b3d60 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.072276 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871b117-c94a-4666-9764-68f4387b3d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.092177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-a871b117-c94a-4666-9764-68f4387b3d60 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:20.093413 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-a871b117-c94a-4666-9764-68f4387b3d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:20.152488 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1419/2796] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:20.160364 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889b88b-0708-4d5c-bd57-d68ae7610ef6 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.160654 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889b88b-0708-4d5c-bd57-d68ae7610ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.160934 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889b88b-0708-4d5c-bd57-d68ae7610ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.180464 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-2889b88b-0708-4d5c-bd57-d68ae7610ef6 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:20.182009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2889b88b-0708-4d5c-bd57-d68ae7610ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:20.189835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1378/2797] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:20.194117 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871b117-c94a-4666-9764-68f4387b3d60 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.204303 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871b117-c94a-4666-9764-68f4387b3d60 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:20.208826 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:20.209410 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.209686 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.209958 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] PATH_INFO: `/v3/domains` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.210623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81530) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:17:20.210926 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:20.212871 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:20.224131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1420/2798] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:17:20.234314 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889b88b-0708-4d5c-bd57-d68ae7610ef6 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.243806 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889b88b-0708-4d5c-bd57-d68ae7610ef6 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:20.248709 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:20.249300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.249574 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.249840 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.250486 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:20.252376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:20.334360 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1379/2799] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:17:20 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:17:20.338973 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.348785 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7ec7-38df-4d6f-9003-c47b14cb491c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:20.375077 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:20.404900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:20.405512 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.405776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.406084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.406748 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:20.409184 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:20.518634 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1421/2800] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:17:20 2022] POST /identity/v3/users => generated 282 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:17:20.522636 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.533417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44c7cf-4110-4aca-963f-ab2a46c7aab4 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:20.538199 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:20.538848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.539129 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.539402 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.540838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:20.542495 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:20.552573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1380/2801] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:20.556989 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.566931 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa1bf5-ef1c-4f9d-b7fc-b437fe64f898 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:20.571900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdb762-5a85-4511-aa13-4d918d7135d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:20.572493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdb762-5a85-4511-aa13-4d918d7135d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.572774 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdb762-5a85-4511-aa13-4d918d7135d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.573045 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdb762-5a85-4511-aa13-4d918d7135d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.573694 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdb762-5a85-4511-aa13-4d918d7135d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:20.575299 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdb762-5a85-4511-aa13-4d918d7135d0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:20.585608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1422/2802] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:20.590207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.599971 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b7f03-569c-4838-8d4c-b3717f64e3f7 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:20.605262 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:20.605902 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.606181 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.606482 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] PATH_INFO: `/v3/projects/b6c6c32cab6945fcbbcf99705322c79c/users/aa93048a3dc141f59adb734670c9acd4/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.621358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c6c32cab6945fcbbcf99705322c79c, user_id=aa93048a3dc141f59adb734670c9acd4, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:20.624332 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:20.699317 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1381/2803] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:17:20 2022] PUT /identity/v3/projects/b6c6c32cab6945fcbbcf99705322c79c/users/aa93048a3dc141f59adb734670c9acd4/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:20.709639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e908-f181-403e-a93d-25fa4de105fd None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.709925 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e908-f181-403e-a93d-25fa4de105fd None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.710287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e908-f181-403e-a93d-25fa4de105fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.737428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-4ae6e908-f181-403e-a93d-25fa4de105fd None None] MFA Rules not processed for user `aa93048a3dc141f59adb734670c9acd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:20.738663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4ae6e908-f181-403e-a93d-25fa4de105fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:20.800939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1423/2804] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:20.805394 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.866066 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c2d058-c3bd-4a15-b203-f69f51f501f0 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:20.870848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:20.871459 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:20.871740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:20.872013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] PATH_INFO: `/v3/projects` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:20.872912 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:20.874839 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:20.966046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1382/2805] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:17:20 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:17:20.970271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e908-f181-403e-a93d-25fa4de105fd None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:20.981122 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e908-f181-403e-a93d-25fa4de105fd None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:21.037594 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:21.038193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.038506 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.038776 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.039397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:21.041623 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:21.119172 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1424/2806] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:17:20 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:17:21.123494 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:21.134867 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65234f-c370-402e-a236-89e95ac028be admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:21.145818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:21.146461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.146744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.147016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.147659 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:21.149274 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:21.160217 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1383/2807] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:21.164418 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:21.176024 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3ab5e-e4a2-46f8-9af8-570277848da0 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:21.185564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:21.185564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.185564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.185564 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] PATH_INFO: `/v3/projects/879ea94b680b4cf5aea84a80aaa6786a/users/8dea17ff34f94381a4d3b0d2f9c7c8b6/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.198192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879ea94b680b4cf5aea84a80aaa6786a, user_id=8dea17ff34f94381a4d3b0d2f9c7c8b6, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:21.200837 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:21.298813 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1425/2808] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:17:21 2022] PUT /identity/v3/projects/879ea94b680b4cf5aea84a80aaa6786a/users/8dea17ff34f94381a4d3b0d2f9c7c8b6/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:21.302831 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:21.313347 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e6a91-ee42-488a-992c-c30aa31a68bf admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:21.333329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:21.369764 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ad35-0b3e-4127-8a1a-f572fad2b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:21.370330 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ad35-0b3e-4127-8a1a-f572fad2b09b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.370610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ad35-0b3e-4127-8a1a-f572fad2b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.370881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ad35-0b3e-4127-8a1a-f572fad2b09b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.371511 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ad35-0b3e-4127-8a1a-f572fad2b09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:21.373344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ad35-0b3e-4127-8a1a-f572fad2b09b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:21.383904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1384/2809] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:21.388440 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:21.393880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:21.399669 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc61f0-c9fd-4a30-8272-a6ba5327f5a3 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:21.404359 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:21.405005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.405289 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.405562 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] PATH_INFO: `/v3/projects/879ea94b680b4cf5aea84a80aaa6786a/users/8dea17ff34f94381a4d3b0d2f9c7c8b6/roles/732ca52314ca470d86d12ac1b06cd16a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.407505 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879ea94b680b4cf5aea84a80aaa6786a, user_id=8dea17ff34f94381a4d3b0d2f9c7c8b6, role_id=732ca52314ca470d86d12ac1b06cd16a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:21.410121 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:21.421519 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1426/2810] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:17:21 2022] PUT /identity/v3/projects/879ea94b680b4cf5aea84a80aaa6786a/users/8dea17ff34f94381a4d3b0d2f9c7c8b6/roles/732ca52314ca470d86d12ac1b06cd16a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:21.433773 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d89d-bc54-4c8b-b3d8-bc61a9dd6a6e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.434053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d89d-bc54-4c8b-b3d8-bc61a9dd6a6e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.434357 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50d89d-bc54-4c8b-b3d8-bc61a9dd6a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.460711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5d50d89d-bc54-4c8b-b3d8-bc61a9dd6a6e None None] MFA Rules not processed for user `8dea17ff34f94381a4d3b0d2f9c7c8b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:21.461894 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5d50d89d-bc54-4c8b-b3d8-bc61a9dd6a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:21.527719 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1385/2811] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:21.538961 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:21.541450 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:21.596575 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5f770-c7a8-4668-82ab-43a2c1180346 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:21.600157 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:21.600742 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.601031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.601286 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.610603 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:21.612005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:21.621762 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1427/2812] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:17:21.817299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50d89d-bc54-4c8b-b3d8-bc61a9dd6a6e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:21.827006 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50d89d-bc54-4c8b-b3d8-bc61a9dd6a6e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:21.830571 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:21.831169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:21.831448 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:21.831736 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:21.889835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:21.891225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:21.900766 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1386/2813] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:17:31.617027 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:31.896037 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=80 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:33.367750 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:33.369794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:33.439923 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a597f53-0f93-4ad5-bb48-f45e6720f4e5 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:33.446873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7739e83-2913-4669-842d-3860c5c25914 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:33.446873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7739e83-2913-4669-842d-3860c5c25914 service placement] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:33.446873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7739e83-2913-4669-842d-3860c5c25914 service placement] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:33.446873 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7739e83-2913-4669-842d-3860c5c25914 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:33.448211 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d7739e83-2913-4669-842d-3860c5c25914 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:33.460071 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7739e83-2913-4669-842d-3860c5c25914 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:33.461893 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7739e83-2913-4669-842d-3860c5c25914 service placement] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:33.472426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1428/2814] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:17:33.721863 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:33.788836 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c9bd4c-b47e-4287-8ebb-29d1260f3ad2 service cinder] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:33.793935 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:33.794588 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:33.794862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:33.795130 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] PATH_INFO: `/v3/users/aa93048a3dc141f59adb734670c9acd4` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:33.796225 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa93048a3dc141f59adb734670c9acd4)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:33.798119 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:33.943080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa93048a3dc141f59adb734670c9acd4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:33.943725 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa93048a3dc141f59adb734670c9acd4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:33.944318 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa93048a3dc141f59adb734670c9acd4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:33.998062 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa93048a3dc141f59adb734670c9acd4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:34.005245 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa93048a3dc141f59adb734670c9acd4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:34.023743 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1387/2815] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:17:33 2022] DELETE /identity/v3/users/aa93048a3dc141f59adb734670c9acd4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:34.028744 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:34.029031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:34.029486 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:34.049214 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:34.050448 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:34.108918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1429/2816] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:34.118609 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:34.120786 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:34.176995 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7eb67e-b87b-47e9-8b8a-7c9dca49c314 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:34.180561 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:34.181157 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:34.182021 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:34.182517 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:34.191295 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:34.192704 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:34.201957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1388/2817] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:17:34.596863 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:34.598369 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:34.612700 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e404285-34c1-46bd-a639-17e1d34ca516 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:34.617423 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:34.618083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:34.618398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:34.618672 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] PATH_INFO: `/v3/projects/b6c6c32cab6945fcbbcf99705322c79c` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:34.620473 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c6c32cab6945fcbbcf99705322c79c)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:34.622636 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:34.726845 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c6c32cab6945fcbbcf99705322c79c', 'request_id': 'req-ec104912-a403-4f16-88df-6f79fd8d675e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:34.732627 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c6c32cab6945fcbbcf99705322c79c', 'request_id': 'req-ec104912-a403-4f16-88df-6f79fd8d675e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:34.738912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c6c32cab6945fcbbcf99705322c79c', 'request_id': 'req-ec104912-a403-4f16-88df-6f79fd8d675e'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:34.824266 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] The token cache is being invalidate because project b6c6c32cab6945fcbbcf99705322c79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:17:34.824552 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:34.825207 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:34.826767 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1430/2818] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:17:34 2022] DELETE /identity/v3/projects/b6c6c32cab6945fcbbcf99705322c79c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:34.830847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:34.840700 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df3526-4961-4ebc-a62b-d88effd23201 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:34.858745 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:34.897414 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:34.898051 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:34.898358 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:34.898630 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] PATH_INFO: `/v3/users/8dea17ff34f94381a4d3b0d2f9c7c8b6` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:34.899731 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dea17ff34f94381a4d3b0d2f9c7c8b6)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:34.901707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:35.100454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dea17ff34f94381a4d3b0d2f9c7c8b6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.101065 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dea17ff34f94381a4d3b0d2f9c7c8b6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.101663 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dea17ff34f94381a4d3b0d2f9c7c8b6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.180644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dea17ff34f94381a4d3b0d2f9c7c8b6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.188573 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-04617781-783b-44c9-9d1b-e0caa457b0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dea17ff34f94381a4d3b0d2f9c7c8b6'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.206452 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1389/2819] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:17:34 2022] DELETE /identity/v3/users/8dea17ff34f94381a4d3b0d2f9c7c8b6 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:35.634066 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:35.636102 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:35.649521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec104912-a403-4f16-88df-6f79fd8d675e admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:35.665666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:35.707792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:35.708425 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:35.708705 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:35.708982 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] PATH_INFO: `/v3/projects/879ea94b680b4cf5aea84a80aaa6786a` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:35.710099 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=879ea94b680b4cf5aea84a80aaa6786a)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:35.712057 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:35.861716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879ea94b680b4cf5aea84a80aaa6786a', 'request_id': 'req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.867187 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879ea94b680b4cf5aea84a80aaa6786a', 'request_id': 'req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.875460 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879ea94b680b4cf5aea84a80aaa6786a', 'request_id': 'req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.946151 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] The token cache is being invalidate because project 879ea94b680b4cf5aea84a80aaa6786a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:17:35.946398 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.947031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-a471a0b8-8b02-4d83-b93d-bac3b0b45ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:35.949100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1431/2820] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:17:35 2022] DELETE /identity/v3/projects/879ea94b680b4cf5aea84a80aaa6786a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:35.954007 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd086d6-605e-46bc-ae60-e85a39e9ce45 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:35.954538 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd086d6-605e-46bc-ae60-e85a39e9ce45 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:35.954667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd086d6-605e-46bc-ae60-e85a39e9ce45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:35.975136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-ecd086d6-605e-46bc-ae60-e85a39e9ce45 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:35.976338 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ecd086d6-605e-46bc-ae60-e85a39e9ce45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:36.035017 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1390/2821] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:36.042884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc653b-ea5c-4830-bf18-4d0aeceb8bc0 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.043033 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc653b-ea5c-4830-bf18-4d0aeceb8bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.043330 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc653b-ea5c-4830-bf18-4d0aeceb8bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.063334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-70dc653b-ea5c-4830-bf18-4d0aeceb8bc0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:36.064622 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-70dc653b-ea5c-4830-bf18-4d0aeceb8bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:36.072567 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1432/2822] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:36.077803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd086d6-605e-46bc-ae60-e85a39e9ce45 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:36.087527 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd086d6-605e-46bc-ae60-e85a39e9ce45 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:36.092172 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:36.092780 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.093063 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.093339 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.093993 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:17:36.094331 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:36.095956 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:36.107430 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1391/2823] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:17:36.111752 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc653b-ea5c-4830-bf18-4d0aeceb8bc0 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:36.121492 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc653b-ea5c-4830-bf18-4d0aeceb8bc0 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:36.126356 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:36.126991 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.127340 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.127616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.128233 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:36.130105 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:36.228114 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1433/2824] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:17:36 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:17:36.232848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:36.242804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526d92e-a2aa-4341-97fd-e1959f5bd748 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:36.268808 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:36.300481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:36.301086 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.301366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.301638 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.302279 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:36.304282 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:36.387100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1392/2825] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:17:36 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:17:36.391387 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:36.402667 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a68fb-25b4-467a-9f88-a8324c225849 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:36.407238 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:36.407864 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.408159 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.408428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.409177 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:36.410784 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:36.421094 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1434/2826] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:36.425535 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:36.435904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43d08e-7d77-4b9b-9efd-ba1305de434e admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:36.440463 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55247851-90ab-408a-84b5-5262c3d0f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:36.441053 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55247851-90ab-408a-84b5-5262c3d0f74f admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.441329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55247851-90ab-408a-84b5-5262c3d0f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.441595 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55247851-90ab-408a-84b5-5262c3d0f74f admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.442355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55247851-90ab-408a-84b5-5262c3d0f74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:36.443921 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55247851-90ab-408a-84b5-5262c3d0f74f admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:36.454686 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1393/2827] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:36.458816 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:36.469307 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db899c53-0977-4400-bab4-de71147e53a7 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:36.473850 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:36.474535 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.474814 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.475089 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] PATH_INFO: `/v3/projects/55375fa2a419488ab6f1f6d5148c405f/users/0ad4dbeb5bc24291abbeef4340358fc4/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.489130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55375fa2a419488ab6f1f6d5148c405f, user_id=0ad4dbeb5bc24291abbeef4340358fc4, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:36.491728 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:36.558116 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1435/2828] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:17:36 2022] PUT /identity/v3/projects/55375fa2a419488ab6f1f6d5148c405f/users/0ad4dbeb5bc24291abbeef4340358fc4/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:36.624740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163800b-caea-4f4c-8b63-076f11fd0588 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.625026 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163800b-caea-4f4c-8b63-076f11fd0588 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.625304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163800b-caea-4f4c-8b63-076f11fd0588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.652476 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-f163800b-caea-4f4c-8b63-076f11fd0588 None None] MFA Rules not processed for user `0ad4dbeb5bc24291abbeef4340358fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:36.653678 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f163800b-caea-4f4c-8b63-076f11fd0588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:36.718549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1394/2829] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:36.729591 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:36.732074 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:36.774680 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:36.809463 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8ac6f-ad85-4038-bac3-e2f66bdc0e49 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:36.813176 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ef418-9dfe-4b33-80c5-e72847df5b80 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990660ce5a9f4213a5b42b3824d0b567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:36.817600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef418-9dfe-4b33-80c5-e72847df5b80 service glance] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:36.817600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef418-9dfe-4b33-80c5-e72847df5b80 service glance] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:36.817600 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ef418-9dfe-4b33-80c5-e72847df5b80 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:36.822500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ef418-9dfe-4b33-80c5-e72847df5b80 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:36.824038 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ef418-9dfe-4b33-80c5-e72847df5b80 service glance] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:36.832698 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1436/2830] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:17:46.716127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:46.864471 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=17 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:55.079292 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163800b-caea-4f4c-8b63-076f11fd0588 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:55.138174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163800b-caea-4f4c-8b63-076f11fd0588 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:55.142644 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:55.143271 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:55.143548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:55.143820 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] PATH_INFO: `/v3/users/0ad4dbeb5bc24291abbeef4340358fc4` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:55.144924 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad4dbeb5bc24291abbeef4340358fc4)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:55.146862 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:55.325353 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad4dbeb5bc24291abbeef4340358fc4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:55.326023 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad4dbeb5bc24291abbeef4340358fc4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:55.326653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad4dbeb5bc24291abbeef4340358fc4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:55.391953 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad4dbeb5bc24291abbeef4340358fc4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:55.399032 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad4dbeb5bc24291abbeef4340358fc4'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:55.416046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1395/2831] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:17:55 2022] DELETE /identity/v3/users/0ad4dbeb5bc24291abbeef4340358fc4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:55.421223 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:55.421718 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:55.421834 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:55.441313 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:55.442551 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:55.501068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1437/2832] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:55.510711 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:55.512202 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:55.567887 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e9114-30e2-4fe7-91f6-67515cb94ad5 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:55.571418 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735bf35-579a-4057-b7ad-d0f1c5bead4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:55.572020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735bf35-579a-4057-b7ad-d0f1c5bead4e service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:55.572297 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735bf35-579a-4057-b7ad-d0f1c5bead4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:55.572567 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735bf35-579a-4057-b7ad-d0f1c5bead4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:55.581563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735bf35-579a-4057-b7ad-d0f1c5bead4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:55.583013 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735bf35-579a-4057-b7ad-d0f1c5bead4e service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:55.592957 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1396/2833] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:17:55.974351 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:55.975932 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:55.990133 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a077ea-9f07-4684-a780-eeed4f0b5024 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:55.995018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:55.995732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:55.996086 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:55.996374 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] PATH_INFO: `/v3/projects/55375fa2a419488ab6f1f6d5148c405f` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:55.997773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] RBAC: Authorizing `identity:delete_project(project_id=55375fa2a419488ab6f1f6d5148c405f)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:55.999794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:56.136195 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55375fa2a419488ab6f1f6d5148c405f', 'request_id': 'req-99d7e511-cf47-42af-89a6-e0989c2b34db'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:56.142579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55375fa2a419488ab6f1f6d5148c405f', 'request_id': 'req-99d7e511-cf47-42af-89a6-e0989c2b34db'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:56.148883 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55375fa2a419488ab6f1f6d5148c405f', 'request_id': 'req-99d7e511-cf47-42af-89a6-e0989c2b34db'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:56.238383 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] The token cache is being invalidate because project 55375fa2a419488ab6f1f6d5148c405f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:17:56.238585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:56.239194 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-99d7e511-cf47-42af-89a6-e0989c2b34db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:17:56.240666 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1438/2834] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:17:55 2022] DELETE /identity/v3/projects/55375fa2a419488ab6f1f6d5148c405f => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:56.245541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ddeaf-793f-430f-b435-0643aa16cf49 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.246039 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ddeaf-793f-430f-b435-0643aa16cf49 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.246138 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ddeaf-793f-430f-b435-0643aa16cf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.265585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-026ddeaf-793f-430f-b435-0643aa16cf49 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:56.266814 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-026ddeaf-793f-430f-b435-0643aa16cf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:56.283986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:56.325596 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1397/2835] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:17:56.333096 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb007-17ef-41f9-b00d-3e11dee91bec None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.333381 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb007-17ef-41f9-b00d-3e11dee91bec None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.333654 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeb007-17ef-41f9-b00d-3e11dee91bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.355256 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-03aeb007-17ef-41f9-b00d-3e11dee91bec None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:56.357188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-03aeb007-17ef-41f9-b00d-3e11dee91bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:56.365018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1439/2836] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:56.369126 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ddeaf-793f-430f-b435-0643aa16cf49 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:56.378878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ddeaf-793f-430f-b435-0643aa16cf49 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:56.383382 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:56.383985 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.384265 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.384537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.385179 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:17:56.385479 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:56.387122 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:56.398491 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1398/2837] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:17:56.403364 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeb007-17ef-41f9-b00d-3e11dee91bec None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:56.412511 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeb007-17ef-41f9-b00d-3e11dee91bec None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:56.417131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:56.417726 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.418642 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.419017 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.419640 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:56.421543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:56.519188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1440/2838] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:17:56 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:17:56.523229 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:56.533795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13802214-4abc-430a-8a83-b09a46e56ecc admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:56.588208 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:56.588803 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.589080 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.589350 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.590625 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:56.592541 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:56.694680 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1399/2839] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:17:56 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:17:56.698952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:56.709877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f659ad-3889-4600-a4f9-a36d1b2af749 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:56.714441 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:56.715648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.715922 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.716191 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.716848 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:56.718444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:56.728495 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1441/2840] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:56.732400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:56.742125 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d384a9-d175-46c6-9fc1-87168a58a39b admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:56.746782 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda59573-7d51-4097-80f5-c9e45ff0e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:56.747629 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda59573-7d51-4097-80f5-c9e45ff0e90b admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.747904 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda59573-7d51-4097-80f5-c9e45ff0e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.748174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda59573-7d51-4097-80f5-c9e45ff0e90b admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.748804 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda59573-7d51-4097-80f5-c9e45ff0e90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:56.750409 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda59573-7d51-4097-80f5-c9e45ff0e90b admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:56.760706 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1400/2841] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:17:56.765175 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:56.774951 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc27a2-ee01-4fe3-84be-308234181cb5 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:56.780323 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:56.780953 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.781229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.781500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] PATH_INFO: `/v3/projects/df60a0e98f294e7382d0b6979a9f3037/users/d0f09815e5bb49a0b92998987ab824ac/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.795524 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df60a0e98f294e7382d0b6979a9f3037, user_id=d0f09815e5bb49a0b92998987ab824ac, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:56.798098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:56.861645 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1442/2842] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:17:56 2022] PUT /identity/v3/projects/df60a0e98f294e7382d0b6979a9f3037/users/d0f09815e5bb49a0b92998987ab824ac/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:17:56.872055 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03af16-b467-46df-8e9e-eb139c486265 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:56.872604 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03af16-b467-46df-8e9e-eb139c486265 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:56.872878 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03af16-b467-46df-8e9e-eb139c486265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:56.900234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-4b03af16-b467-46df-8e9e-eb139c486265 None None] MFA Rules not processed for user `d0f09815e5bb49a0b92998987ab824ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:17:56.901437 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-4b03af16-b467-46df-8e9e-eb139c486265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:56.964389 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1401/2843] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:17:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:17:56.975615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:17:56.977288 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:17:56.999665 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:17:57.034759 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eba03-37d0-47d4-b4d9-07f7855debe8 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:17:57.038366 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f525948-48e6-49da-9ef4-000fe6aec0a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:17:57.041608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f525948-48e6-49da-9ef4-000fe6aec0a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:17:57.041608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f525948-48e6-49da-9ef4-000fe6aec0a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:17:57.041608 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f525948-48e6-49da-9ef4-000fe6aec0a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:17:57.047659 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f525948-48e6-49da-9ef4-000fe6aec0a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:17:57.049069 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f525948-48e6-49da-9ef4-000fe6aec0a2 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:17:57.057787 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1443/2844] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:17:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:18:06.961337 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:07.089639 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=17 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:21.162442 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03af16-b467-46df-8e9e-eb139c486265 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:21.225847 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03af16-b467-46df-8e9e-eb139c486265 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:21.230344 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:21.230982 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:21.231263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:21.231540 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] PATH_INFO: `/v3/users/d0f09815e5bb49a0b92998987ab824ac` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:21.232656 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f09815e5bb49a0b92998987ab824ac)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:21.234923 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:21.421933 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f09815e5bb49a0b92998987ab824ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:21.422534 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f09815e5bb49a0b92998987ab824ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:21.423136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f09815e5bb49a0b92998987ab824ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:21.488361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f09815e5bb49a0b92998987ab824ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:21.495653 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f09815e5bb49a0b92998987ab824ac'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:21.513137 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1402/2845] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:18:21 2022] DELETE /identity/v3/users/d0f09815e5bb49a0b92998987ab824ac => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:18:21.518306 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:21.518939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:21.518939 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:21.537761 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:21.539001 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:21.597378 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1444/2846] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:21.607044 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:21.608513 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:21.664733 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec380681-9a52-47ad-af74-e322d153d701 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:21.668299 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dddaa9-b539-47b8-bfdb-37b7539fdd04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:21.668896 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dddaa9-b539-47b8-bfdb-37b7539fdd04 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:21.669171 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dddaa9-b539-47b8-bfdb-37b7539fdd04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:21.669443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dddaa9-b539-47b8-bfdb-37b7539fdd04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:21.678447 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dddaa9-b539-47b8-bfdb-37b7539fdd04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:21.679795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dddaa9-b539-47b8-bfdb-37b7539fdd04 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:21.689451 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1403/2847] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:18:22.085019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:22.086521 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:22.098451 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643b16e-927b-4e0f-8848-f8db20e25f2f None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:22.103009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:22.103638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.103920 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.104199 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] PATH_INFO: `/v3/projects/df60a0e98f294e7382d0b6979a9f3037` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.105468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] RBAC: Authorizing `identity:delete_project(project_id=df60a0e98f294e7382d0b6979a9f3037)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:22.107468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:22.280633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df60a0e98f294e7382d0b6979a9f3037', 'request_id': 'req-48bdc31a-2f47-4fb7-99fe-13970c6c262b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:22.285985 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df60a0e98f294e7382d0b6979a9f3037', 'request_id': 'req-48bdc31a-2f47-4fb7-99fe-13970c6c262b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:22.292258 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df60a0e98f294e7382d0b6979a9f3037', 'request_id': 'req-48bdc31a-2f47-4fb7-99fe-13970c6c262b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:22.382221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] The token cache is being invalidate because project df60a0e98f294e7382d0b6979a9f3037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:18:22.382546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:22.383165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-48bdc31a-2f47-4fb7-99fe-13970c6c262b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:22.384648 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1445/2848] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:18:22 2022] DELETE /identity/v3/projects/df60a0e98f294e7382d0b6979a9f3037 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:18:22.389557 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c49077-75dc-4f42-be0e-cb23f7e6985e None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.390100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c49077-75dc-4f42-be0e-cb23f7e6985e None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.390100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c49077-75dc-4f42-be0e-cb23f7e6985e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.409637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-19c49077-75dc-4f42-be0e-cb23f7e6985e None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:22.411009 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-19c49077-75dc-4f42-be0e-cb23f7e6985e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:22.428343 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:22.470461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1404/2849] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:22.477999 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94afca-aa12-4613-b7be-6b3eaf4b4d3d None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.478314 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94afca-aa12-4613-b7be-6b3eaf4b4d3d None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.478592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94afca-aa12-4613-b7be-6b3eaf4b4d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.499100 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-9c94afca-aa12-4613-b7be-6b3eaf4b4d3d None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:22.500377 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9c94afca-aa12-4613-b7be-6b3eaf4b4d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:22.508110 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1446/2850] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:22.512213 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c49077-75dc-4f42-be0e-cb23f7e6985e None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:22.521795 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c49077-75dc-4f42-be0e-cb23f7e6985e None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:22.526818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:22.527415 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.527690 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.527955 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.528610 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:18:22.528892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:22.531030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:22.542417 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1405/2851] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:18:22.546725 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94afca-aa12-4613-b7be-6b3eaf4b4d3d None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:22.555912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94afca-aa12-4613-b7be-6b3eaf4b4d3d None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:22.560629 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:22.561779 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.562126 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.562428 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.563040 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:22.565009 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:22.684854 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1447/2852] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:18:22 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:18:22.685990 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:22.696669 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d1399-455f-4738-a838-c0ea8831f8a4 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:22.753324 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:22.753861 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.754140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.754443 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.755087 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:22.756986 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:22.834552 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1406/2853] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:18:22 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:18:22.838836 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:22.849287 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61879a-8b87-4d3c-bb22-ca3166ff1902 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:22.853950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:22.854579 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.854912 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.855192 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.855867 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:22.857468 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:22.868400 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1448/2854] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:18:22.878613 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:22.888677 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc8e6-0751-4fc2-8f6c-e69382f34558 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:22.893556 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4f886-d4cb-40ae-ba43-ac42c419b6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:22.894152 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4f886-d4cb-40ae-ba43-ac42c419b6d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.894461 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4f886-d4cb-40ae-ba43-ac42c419b6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.894735 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4f886-d4cb-40ae-ba43-ac42c419b6d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.895374 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4f886-d4cb-40ae-ba43-ac42c419b6d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:22.896976 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4f886-d4cb-40ae-ba43-ac42c419b6d4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:22.907566 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1407/2855] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:18:22.911660 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:22.922098 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93182d89-079f-4273-95e5-fc437aed0118 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:22.926739 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:22.927393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:22.927663 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:22.927936 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] PATH_INFO: `/v3/projects/a4bc4b6cef244d51a67e83d81b785605/users/dbdaa978e7874354962ba13c8fa58f95/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:22.943008 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4bc4b6cef244d51a67e83d81b785605, user_id=dbdaa978e7874354962ba13c8fa58f95, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:22.945633 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:23.023193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1449/2856] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:18:22 2022] PUT /identity/v3/projects/a4bc4b6cef244d51a67e83d81b785605/users/dbdaa978e7874354962ba13c8fa58f95/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:18:23.033444 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325f993-b6bc-4e14-ab8a-2b88751e4d65 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:23.041300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325f993-b6bc-4e14-ab8a-2b88751e4d65 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:23.041300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325f993-b6bc-4e14-ab8a-2b88751e4d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:23.062085 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-e325f993-b6bc-4e14-ab8a-2b88751e4d65 None None] MFA Rules not processed for user `dbdaa978e7874354962ba13c8fa58f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:23.063376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e325f993-b6bc-4e14-ab8a-2b88751e4d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:23.125856 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1408/2857] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:23.142261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:23.143917 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:23.165583 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:23.199713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941984f-398c-4cea-a928-b7f1e8397e5c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:23.203397 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c21ba-eb43-4264-a006-7aca41155bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:23.207499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c21ba-eb43-4264-a006-7aca41155bea service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:23.207499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c21ba-eb43-4264-a006-7aca41155bea service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:23.207499 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c21ba-eb43-4264-a006-7aca41155bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:23.212530 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c21ba-eb43-4264-a006-7aca41155bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:23.213959 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c21ba-eb43-4264-a006-7aca41155bea service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:23.223346 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1450/2858] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:18:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:18:27.211436 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325f993-b6bc-4e14-ab8a-2b88751e4d65 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:27.229979 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:27.274312 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325f993-b6bc-4e14-ab8a-2b88751e4d65 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:27.278846 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:27.279478 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:27.279823 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:27.280102 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] PATH_INFO: `/v3/users/dbdaa978e7874354962ba13c8fa58f95` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:27.281212 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbdaa978e7874354962ba13c8fa58f95)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:27.283165 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:27.487207 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbdaa978e7874354962ba13c8fa58f95'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:27.489503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbdaa978e7874354962ba13c8fa58f95'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:27.489503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbdaa978e7874354962ba13c8fa58f95'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:27.565563 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbdaa978e7874354962ba13c8fa58f95'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:27.572585 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbdaa978e7874354962ba13c8fa58f95'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:27.590147 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1409/2859] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:18:27 2022] DELETE /identity/v3/users/dbdaa978e7874354962ba13c8fa58f95 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:18:27.595437 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:27.595909 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:27.596010 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:27.616918 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:27.619638 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:27.682312 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1451/2860] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:27.692631 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:27.694024 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:27.751838 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13476b6-689c-420d-9d57-be04a57fa999 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:27.755354 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d87ce1-fb5c-4065-9f49-f92e69602bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:27.755954 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87ce1-fb5c-4065-9f49-f92e69602bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:27.756234 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87ce1-fb5c-4065-9f49-f92e69602bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:27.756506 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d87ce1-fb5c-4065-9f49-f92e69602bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:27.765871 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d87ce1-fb5c-4065-9f49-f92e69602bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:27.767400 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d87ce1-fb5c-4065-9f49-f92e69602bc5 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:27.776601 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1410/2861] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:18:28.172915 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:28.174404 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:28.186500 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d44f6-9af3-4a6d-a930-a89542fa1795 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:28.191261 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:28.191900 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.192188 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.192443 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] PATH_INFO: `/v3/projects/a4bc4b6cef244d51a67e83d81b785605` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.193737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4bc4b6cef244d51a67e83d81b785605)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:28.195727 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:28.325252 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4bc4b6cef244d51a67e83d81b785605', 'request_id': 'req-6bb3f855-686b-43c7-b818-bcc1407ae174'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:28.331146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4bc4b6cef244d51a67e83d81b785605', 'request_id': 'req-6bb3f855-686b-43c7-b818-bcc1407ae174'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:28.337179 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4bc4b6cef244d51a67e83d81b785605', 'request_id': 'req-6bb3f855-686b-43c7-b818-bcc1407ae174'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:28.402028 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] The token cache is being invalidate because project a4bc4b6cef244d51a67e83d81b785605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:18:28.402343 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:28.402952 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-6bb3f855-686b-43c7-b818-bcc1407ae174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:28.404409 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1452/2862] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:18:28 2022] DELETE /identity/v3/projects/a4bc4b6cef244d51a67e83d81b785605 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:18:28.409454 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99057ae5-54b6-4bbb-ad2e-af220aa41245 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.410429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99057ae5-54b6-4bbb-ad2e-af220aa41245 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.410429 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99057ae5-54b6-4bbb-ad2e-af220aa41245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.430356 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-99057ae5-54b6-4bbb-ad2e-af220aa41245 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:28.431549 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-99057ae5-54b6-4bbb-ad2e-af220aa41245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:28.465355 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:28.490131 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1411/2863] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:28.500160 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a4a59-359c-4df5-af81-ba2e58533bb0 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.500442 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a4a59-359c-4df5-af81-ba2e58533bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.500713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a4a59-359c-4df5-af81-ba2e58533bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.520605 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-261a4a59-359c-4df5-af81-ba2e58533bb0 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:28.521794 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-261a4a59-359c-4df5-af81-ba2e58533bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:28.529616 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1453/2864] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:28.533791 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99057ae5-54b6-4bbb-ad2e-af220aa41245 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:28.543329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99057ae5-54b6-4bbb-ad2e-af220aa41245 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:28.548335 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:28.548941 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.549221 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.549546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.550194 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81529) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 22 11:18:28.550546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:28.552263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:28.564632 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1412/2865] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 22 11:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 22 11:18:28.568769 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a4a59-359c-4df5-af81-ba2e58533bb0 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:28.578806 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a4a59-359c-4df5-af81-ba2e58533bb0 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:28.583302 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:28.583904 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.584178 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.584447 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] PATH_INFO: `/v3/projects` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.585060 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:28.586968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:28.666747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1454/2866] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 22 11:18:28 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 22 11:18:28.671395 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:28.681296 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b0fbd-37b5-4487-9dc7-9570740f12b4 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:28.738376 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:28.738983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.739263 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.739537 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.740149 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:28.742040 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:28.812366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1413/2867] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 22 11:18:28 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 22 11:18:28.816311 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:28.826706 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab392fd9-b135-4320-a6a3-ecdf156d55be admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:28.831393 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:28.831933 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.832208 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.832475 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.833108 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:28.834737 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:28.844544 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1455/2868] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:18:28.848481 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:28.859637 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719e1f2-eebf-4c7b-956a-5a4c6a4a6a88 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:28.863816 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfed6a-371b-41e7-8891-c34864faead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:28.864423 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfed6a-371b-41e7-8891-c34864faead7 admin admin] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.864703 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfed6a-371b-41e7-8891-c34864faead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.864977 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfed6a-371b-41e7-8891-c34864faead7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.865624 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfed6a-371b-41e7-8891-c34864faead7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:28.867260 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfed6a-371b-41e7-8891-c34864faead7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:28.879077 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1414/2869] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 22 11:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 22 11:18:28.883225 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:28.893291 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897b6cc-3842-46f6-9217-0ff11a61baaa admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:28.897878 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:28.898541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:28.898817 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:28.899084 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] PATH_INFO: `/v3/projects/ad6ff5a309d244f9ac20399bff4dcc27/users/aadc4ee42aa14dff8860e7dc84ff4e0b/roles/f4ba3d348adb4b6497d8a69e88e3822b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:28.901235 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:28.915226 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6ff5a309d244f9ac20399bff4dcc27, user_id=aadc4ee42aa14dff8860e7dc84ff4e0b, role_id=f4ba3d348adb4b6497d8a69e88e3822b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:28.919507 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:29.021532 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1456/2870] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 22 11:18:28 2022] PUT /identity/v3/projects/ad6ff5a309d244f9ac20399bff4dcc27/users/aadc4ee42aa14dff8860e7dc84ff4e0b/roles/f4ba3d348adb4b6497d8a69e88e3822b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:18:29.032488 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1981d0-dfd7-4829-bcb1-2cd14718c6c3 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:29.032788 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1981d0-dfd7-4829-bcb1-2cd14718c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:29.033096 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1981d0-dfd7-4829-bcb1-2cd14718c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:29.059752 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-7c1981d0-dfd7-4829-bcb1-2cd14718c6c3 None None] MFA Rules not processed for user `aadc4ee42aa14dff8860e7dc84ff4e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:29.060969 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-7c1981d0-dfd7-4829-bcb1-2cd14718c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:29.124277 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1415/2871] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:18:29.134820 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:29.136534 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:29.193193 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336128-39b2-45c7-b20a-5231ef2cd1a6 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:29.196825 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7cdb62860b6243bb84c31d1caff4bc77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:29.197427 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:29.197713 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:29.197983 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:29.206710 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:29.208130 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:29.216835 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1457/2872] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:18:31.009067 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929394e7-89df-4146-a551-f88b1046a03b None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:31.009594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929394e7-89df-4146-a551-f88b1046a03b None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:31.009594 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929394e7-89df-4146-a551-f88b1046a03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:31.027951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-929394e7-89df-4146-a551-f88b1046a03b None None] MFA Rules not processed for user `aadc4ee42aa14dff8860e7dc84ff4e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:31.029136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-929394e7-89df-4146-a551-f88b1046a03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:31.035835 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1416/2873] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:39.212847 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:41.022828 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:44.391981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:44.393571 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:44.479355 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5405f7e-b447-4e3a-82c6-19fc1a2496b5 service cinder] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:44.485940 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f5fe23bc8294bc3861e0db9dad2045e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:44.487154 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:44.487543 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:44.487965 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:44.489956 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:44.503050 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:44.504619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:44.515466 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1458/2874] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:18:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:18:50.449127 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db62c5-aa28-4255-b02a-82d17fba6b35 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:50.449648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db62c5-aa28-4255-b02a-82d17fba6b35 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:50.449648 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db62c5-aa28-4255-b02a-82d17fba6b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:50.508881 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-d2db62c5-aa28-4255-b02a-82d17fba6b35 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:50.510069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-d2db62c5-aa28-4255-b02a-82d17fba6b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:50.572250 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1417/2875] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:50.718747 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:50.720977 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:50.809758 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19c7fc-6fa7-4e25-874a-3ce76f27f233 service nova] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:50.815673 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:50.816326 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:50.816607 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:50.816877 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:50.825005 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:50.826611 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:50.834905 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1459/2876] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:18:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:18:51.641532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5bf41-eb85-4ae3-930a-2b5dadbd5e54 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:51.641983 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5bf41-eb85-4ae3-930a-2b5dadbd5e54 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:51.642169 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5bf41-eb85-4ae3-930a-2b5dadbd5e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:51.660830 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-85f5bf41-eb85-4ae3-930a-2b5dadbd5e54 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:51.662016 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-85f5bf41-eb85-4ae3-930a-2b5dadbd5e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:51.669100 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1418/2877] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 22 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:51.681046 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:51.692267 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5aaad0-3991-4074-95ad-86c310a360d2 service swift] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:51.697205 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:51.697792 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:51.698068 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:51.698372 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:51.706732 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:51.708277 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:51.716031 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1460/2878] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:18:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:18:52.424191 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93292-0fed-4545-9d95-8510a64e0d63 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:52.424474 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93292-0fed-4545-9d95-8510a64e0d63 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:52.424759 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93292-0fed-4545-9d95-8510a64e0d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:52.444646 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-9ad93292-0fed-4545-9d95-8510a64e0d63 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:52.445936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-9ad93292-0fed-4545-9d95-8510a64e0d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:52.453620 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1419/2879] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 22 11:18:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:52.465546 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:52.476520 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e42b7c-83a4-46a7-9929-225ce3db945b service swift] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:52.481464 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:52.482051 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:52.482422 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:52.482697 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:52.490780 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:52.492386 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:52.501165 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1461/2880] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:18:53.137749 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295066f-6c3a-490d-b260-bae52eb80d00 None None] REQUEST_METHOD: `POST` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:53.138223 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295066f-6c3a-490d-b260-bae52eb80d00 None None] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:53.138379 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295066f-6c3a-490d-b260-bae52eb80d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:53.158605 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.auth.core [None req-5295066f-6c3a-490d-b260-bae52eb80d00 None None] MFA Rules not processed for user `4047588f18044f6da1c9884d5e6ec0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81529) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:53.159777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-5295066f-6c3a-490d-b260-bae52eb80d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:53.166744 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1420/2881] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 22 11:18:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 22 11:18:53.179615 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:53.193257 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39cfb5-ede0-4989-b087-ac6ca8c11a34 service swift] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:53.198334 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f2a65-ddc0-4d11-83f8-a5eb957b3c5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5e1b8292ac84020b6ac6db471a6e0ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:53.198942 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f2a65-ddc0-4d11-83f8-a5eb957b3c5a service swift] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:53.199222 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f2a65-ddc0-4d11-83f8-a5eb957b3c5a service swift] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:53.199493 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f2a65-ddc0-4d11-83f8-a5eb957b3c5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:53.208309 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f2a65-ddc0-4d11-83f8-a5eb957b3c5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:53.209874 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f2a65-ddc0-4d11-83f8-a5eb957b3c5a service swift] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:53.218012 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1462/2882] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 22 11:18:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 22 11:18:55.999484 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295066f-6c3a-490d-b260-bae52eb80d00 None None] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:56.060892 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295066f-6c3a-490d-b260-bae52eb80d00 None None] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:56.065858 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:56.066545 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:56.066800 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:56.067069 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] PATH_INFO: `/v3/users/aadc4ee42aa14dff8860e7dc84ff4e0b` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:56.068189 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadc4ee42aa14dff8860e7dc84ff4e0b)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:56.070095 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:56.234946 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:56.236781 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadc4ee42aa14dff8860e7dc84ff4e0b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:56.237450 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadc4ee42aa14dff8860e7dc84ff4e0b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:56.238043 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadc4ee42aa14dff8860e7dc84ff4e0b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:56.308329 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadc4ee42aa14dff8860e7dc84ff4e0b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:56.316829 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadc4ee42aa14dff8860e7dc84ff4e0b'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:56.335109 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1421/2883] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:18:55 2022] DELETE /identity/v3/users/aadc4ee42aa14dff8860e7dc84ff4e0b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:18:56.339733 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] REQUEST_METHOD: `POST` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:56.340227 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:56.340328 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:56.364185 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.auth.core [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] MFA Rules not processed for user `5f27905b35c14e3c9541d9819a2b2185`. Rule list: `[]` (Enabled: `True`). {{(pid=81530) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 22 11:18:56.365968 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:56.380131 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=100 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:18:56.440322 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1463/2884] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 22 11:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 22 11:18:56.450034 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:56.451532 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:56.512493 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4825-ab6a-4020-a43f-0d0a528c668c admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:56.516390 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:56.517030 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:56.517311 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:56.517581 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:56.526705 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:56.528070 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:56.537707 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1422/2885] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:18:56.815950 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:18:56.817384 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:18:56.830619 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a49b4-d5e9-43a5-bc9c-c59a5e6fc422 None None] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:18:56.835716 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:18:56.836542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:18:56.837019 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:18:56.837857 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] PATH_INFO: `/v3/projects/ad6ff5a309d244f9ac20399bff4dcc27` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:18:56.839300 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6ff5a309d244f9ac20399bff4dcc27)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:18:56.841880 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:18:56.998431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6ff5a309d244f9ac20399bff4dcc27', 'request_id': 'req-4c49b6f7-bafe-4704-a84a-586244a2b5ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:57.004229 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6ff5a309d244f9ac20399bff4dcc27', 'request_id': 'req-4c49b6f7-bafe-4704-a84a-586244a2b5ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:57.011482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6ff5a309d244f9ac20399bff4dcc27', 'request_id': 'req-4c49b6f7-bafe-4704-a84a-586244a2b5ae'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:57.085461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] The token cache is being invalidate because project ad6ff5a309d244f9ac20399bff4dcc27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81530) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:18:57.085461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:57.085461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:18:57.085461 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1464/2886] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:18:56 2022] DELETE /identity/v3/projects/ad6ff5a309d244f9ac20399bff4dcc27 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:19:02.488114 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:19:02.489602 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:02.502046 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488662bb-95e3-4036-b07d-49a0c2ef189e service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:19:02.548667 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edd00e00b48641c8bcaacaaa7c2dd149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:19:02.549267 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:19:02.549546 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:19:02.549818 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:19:02.551548 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:02.611936 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:19:02.613359 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:19:02.623140 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1423/2887] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:19:06.163963 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:19:06.165542 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:06.245980 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49b6f7-bafe-4704-a84a-586244a2b5ae admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:19:06.252566 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:19:06.256549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:19:06.256549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:19:06.256549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:19:06.256549 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:06.269181 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:19:06.270889 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:19:06.283592 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1465/2888] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:19:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:19:09.781926 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:19:09.792974 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911d662-2dbc-449e-aaf4-274b8b60d622 service neutron] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:19:09.798753 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:19:09.799368 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] REQUEST_METHOD: `GET` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:19:09.799650 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:19:09.799943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:19:09.801777 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:09.813682 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:19:09.815361 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:19:09.825300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1424/2889] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:19:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 22 11:19:09.900768 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:19:09.902217 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:09.917083 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ae35c-6c2f-4468-8694-35621e87a733 service placement] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:19:09.922260 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:19:09.922881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:19:09.923161 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:19:09.923431 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] PATH_INFO: `/v3/users/9ca8c5d1971a4ec1a448f2081ceb835b` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:19:09.924541 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca8c5d1971a4ec1a448f2081ceb835b)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:19:09.926573 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:19:09.950823 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:19:10.064482 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca8c5d1971a4ec1a448f2081ceb835b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.065142 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca8c5d1971a4ec1a448f2081ceb835b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.065770 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca8c5d1971a4ec1a448f2081ceb835b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.072902 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca8c5d1971a4ec1a448f2081ceb835b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.144582 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca8c5d1971a4ec1a448f2081ceb835b'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.162699 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1466/2890] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:19:09 2022] DELETE /identity/v3/users/9ca8c5d1971a4ec1a448f2081ceb835b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:19:10.434740 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:19:10.440664 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:19:10.447278 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d034ea-cff3-43c8-9856-171c864ab08a service placement] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:19:10.506307 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:19:10.506951 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:19:10.507231 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:19:10.507503 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] PATH_INFO: `/v3/projects/02c34172aa804fe695944f629a348c58` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:19:10.508623 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c34172aa804fe695944f629a348c58)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:19:10.510553 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:19:10.614848 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c34172aa804fe695944f629a348c58', 'request_id': 'req-6e93f128-a019-4283-8bb1-69ea8fb50ea7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.620174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c34172aa804fe695944f629a348c58', 'request_id': 'req-6e93f128-a019-4283-8bb1-69ea8fb50ea7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.626666 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c34172aa804fe695944f629a348c58', 'request_id': 'req-6e93f128-a019-4283-8bb1-69ea8fb50ea7'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.716300 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] The token cache is being invalidate because project 02c34172aa804fe695944f629a348c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:19:10.720176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.720176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:10.720176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1425/2891] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:19:10 2022] DELETE /identity/v3/projects/02c34172aa804fe695944f629a348c58 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:19:10.723570 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:19:10.724981 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:10.738429 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73aedb8-0732-461f-ae21-c2372b95bd41 admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:19:10.794221 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:19:10.794870 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:19:10.795146 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:19:10.795417 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] PATH_INFO: `/v3/users/699570a9d2ab42da873fca852e4f6fd4` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:19:10.796682 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] RBAC: Authorizing `identity:delete_user(user_id=699570a9d2ab42da873fca852e4f6fd4)` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:19:10.798675 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:19:11.021881 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '699570a9d2ab42da873fca852e4f6fd4'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.022773 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '699570a9d2ab42da873fca852e4f6fd4'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.023455 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '699570a9d2ab42da873fca852e4f6fd4'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.030892 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '699570a9d2ab42da873fca852e4f6fd4'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.101426 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.notifications [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '699570a9d2ab42da873fca852e4f6fd4'} {{(pid=81530) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.118929 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1467/2892] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 22 11:19:10 2022] DELETE /identity/v3/users/699570a9d2ab42da873fca852e4f6fd4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:19:11.360176 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Authenticating user token {{(pid=81529) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:19:11.365141 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.fernet_utils [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81529) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:19:11.374970 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f128-a019-4283-8bb1-69ea8fb50ea7 admin admin] Validating token access rules against request {{(pid=81529) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:19:11.432089 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f27905b35c14e3c9541d9819a2b2185', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1b87adbe93482a946d0fcf7c513809', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81529) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:19:11.432736 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:19:11.433020 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:19:11.433293 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] PATH_INFO: `/v3/projects/77c4ea94394e48aeb13f7e5cc391f0ba` {{(pid=81529) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:19:11.434467 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77c4ea94394e48aeb13f7e5cc391f0ba)` {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:19:11.436366 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] RBAC: Authorization granted {{(pid=81529) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:19:11.536107 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:19:11.545304 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77c4ea94394e48aeb13f7e5cc391f0ba', 'request_id': 'req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.551174 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77c4ea94394e48aeb13f7e5cc391f0ba', 'request_id': 'req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.557185 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77c4ea94394e48aeb13f7e5cc391f0ba', 'request_id': 'req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51'} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.669734 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] The token cache is being invalidate because project 77c4ea94394e48aeb13f7e5cc391f0ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81529) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 22 11:19:11.669943 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.670579 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG keystone.notifications [None req-8b9f5b8d-6c76-4bfb-964a-1e163eedcf51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81529) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 22 11:19:11.672136 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: [pid: 81529|app: 0|req: 1426/2893] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 22 11:19:11 2022] DELETE /identity/v3/projects/77c4ea94394e48aeb13f7e5cc391f0ba => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 22 11:19:21.115576 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=73,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:19:21.733736 dv-cinder-853941-1-iscsi devstack@keystone.service[81529]: DEBUG dbcounter [-] [81529] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81529) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 22 11:22:39.365522 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Authenticating user token {{(pid=81530) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 22 11:22:39.367048 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:22:39.505430 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b383d73-2ee5-41cf-883b-af7a03b5d56c admin admin] Validating token access rules against request {{(pid=81530) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 22 11:22:39.564271 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862a6bd-2f24-4cb9-a728-c49e2bca3c9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3d2213b387f4938ae20eeb16c9352de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e46a0cfe4c2b4edf9d5e27161f97e08c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81530) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 22 11:22:39.564884 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862a6bd-2f24-4cb9-a728-c49e2bca3c9d service placement] REQUEST_METHOD: `GET` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 22 11:22:39.565168 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862a6bd-2f24-4cb9-a728-c49e2bca3c9d service placement] SCRIPT_NAME: `/identity` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 22 11:22:39.565444 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862a6bd-2f24-4cb9-a728-c49e2bca3c9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81530) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 22 11:22:39.567186 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.fernet_utils [None req-5862a6bd-2f24-4cb9-a728-c49e2bca3c9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81530) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 22 11:22:39.579018 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862a6bd-2f24-4cb9-a728-c49e2bca3c9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 22 11:22:39.580585 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862a6bd-2f24-4cb9-a728-c49e2bca3c9d service placement] RBAC: Authorization granted {{(pid=81530) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 22 11:22:39.592156 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: [pid: 81530|app: 0|req: 1468/2894] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 22 11:22:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 22 11:22:49.587906 dv-cinder-853941-1-iscsi devstack@keystone.service[81530]: DEBUG dbcounter [-] [81530] Writing DB stats keystone:SELECT=89 {{(pid=81530) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}